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ABSTRACT 

This research discusses a program which aids the 
user of an automatic programming system CAPS) In the 
"debugging** of his model of his pro&lem situation. In 
essence, the user must make sure that he and the APS JB&2&. 
th» same thing by the description of the problem which the 
APS Is to solve. The problem domain considered In this 
thesis fs that of "business games* (l.e,, the management 
simulation games which are used as a teaming tool In the 
study of management). A language for describing models of 
these game* Is presented. The paper the** describes the 
program's methods of simulating and finding bugs In models 
written In ttils language. Important aspects of the program's 
problem* so! vTng approach to debugging mr&- Its Interna! 
knowledge of ' '•bugs'* and of user intention within the model. 
This Interna:! knowledge stresses the fmp^rtsfice of bugs 
arising from the tnteractlon of suimodels wlttrln the model . 
Some details of the program's Impleawwtatlon (In the 
Connive r language) are discussed. The necessity of 
"mod*! -debugging" In automatic programming Is emphasized. 
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1 Introduction 

The purpose of this research Is to 
explore a methodology for debugging certain models of real 
world situations. The models considered here consist of 
groups of well-defined submodels. The submodels themselves 
are fairly structured; the Interaction between submodels Is 
not. In this paper I will discuss a program which uses the 
techniques of goal -programming to explore the Interactive 
behavior of a given model. The basic Idea Is that a bug In 
the model will give rise to a "problem". The program then 
tries to solve this problem in an environment defined and 
constrained by the model. Those steps at which the 
program's problem-solving process encounters constraints 
caused by unintended Interaction of submodels suggest 
possible locations of bugs within the model. 

To a large extent, the problems of this 
research are "artificial Intelligence" problems. That Is, 
the research problems Involve representation of knowledge In 
a form which is useful to the problem-solver, and 
representation of the problem-solving process as a computer 
program. The remainder of this paper will deal with one 
solution of these problems for a program which debugs models 
of management situations. This section will nrpwnt a mnro 
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complete explanation of the area of model -debugging as I see 
It. The next section provides an overview of the whole 
debugging process In the context of a detailed example. 
Later sections develop some Ideas about bugs, 
problem-solving, goal -programming, and the program Itself. 
1.1 Define "define" 

1.1.1 What Is. a rodei? 

Marvin MInsky describes the concept of a 
"model" as follows: 

If a creature can answer a question 
about a hypothetical experiment without actually 
performing It, then It has demonstrated some 
knowledge about the world. For his answer to the 
question must be an encoded description of the 
behavior (Inside the creature) of a sub-machine or 
"model" responding to an encoded description of 
the world situation described by the question. 

We use the term "model" In the following 
sense: to an observer B, an object A* Is a model 
of an object A to the extent that B can use A* to 
answer questions that Interest htm about A. |12| 

For the purpose of this research, the term "model" will be 
used In a much less general and more concrete way. 
Specifically, the program discussed here requires that the 
"encoded description" be of a particular pre-defined type, 
that the kinds of world-objects "A" to be modelled belong to 
a very limited class of things, and that "B ul s questions of 
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Interest be sharply restricted. 

After this section, the term "model" 
will be used to refer to a user-defined collection of 
constructs In a model specification language (MSL) 
(presented In section «*.l) which describes a "real-world" 
management system. (1) For now, suffice It to say that a 
"model" Is a user's description of his system of Interest. 
That Is, the user thinks that the model describes his 
system — actually, the model contains bugs. 

1.1.2 What Is. debugging? 

When a model's performance Is not what 
the user expects, we say that the model has a "bug" (see 
section 3). The process of finding what causes the 
discrepancy between performance and expectation Is called 
"debugging". It Is the nature of complex processes that the 
cause of a discrepancy may be related to the manifestation 
of the discrepancy only through a seemingly Intricate chain 
of reasoning. The purpose of this research Is to write a 
program which knows the necessary kind of reasoning to go 
from the manifestation to the cause of a bug. 



(1) 

Actually, a real-world business game . 
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In order to Incorporate this reasoning 
process, the program must have knowledge about MSL models 
(see k.l), the kinds of bugs that occur In MSL models (see 
3.3), how these bugs manifest themselves (see fc.U.2), and 
how the causes are related to the manifestations (see 
k.k. 3). Of course, this Is In some sense the "whole story"; 
before launching Into It, It might be a good Idea to examine 
our reasons for worrying about model -debugging In the first 
place. 

1.2 Jhs. Importance o£ fflfldfil-debUgglng 
1.2.1 MorifiT-debuggIng as a universal gancfigJL 

The process of gaining knowledge about 
the world Is a process of model formation and debugging. 
The progress of all organized thought, especially science, 
has often been described In this way. More recently, work 
by psychologists such as Plaget and artificial Intelligence 
researchers such as Seymour Papert has brought this model 
formation/debugging view to bear on the entire learning 
process. Certainly, no one can doubt the Importance of 
studying so fundamental a process. 

Of course, In this research, the 
viewpoint must be strictly limited. The following sections 
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will describe a process which seems only barely related to 
the grandiose exaltations of the previous paragraph. For 
one thing, the extremely close Interaction between model 
debugging and formation will be greatly restricted to allow 
examination of the debugging process Itself. Also, the 
restrictions Inherent (1) In the "show a working program" 
approach of this research make the class of problems seem 
trivial when compared to the overall problem of 
model -debugging. 

Although I could now claim that the 
validity of this research effort Is that It provides an 
Initial Investigation Into a very hairy area (the usual 
Induction step ?n artificial Intelligence theses), I will 
move In more practical directions. (Of course, I hope for 
the higher parallels all along.) Specifically, I consider 
the Importance of the kind of model-debugging actually 
presented here for the new field of automatic programming. 

1.2.2 Model -debugging in. automatic programming 



(1) These restrictions are "Inherent" at this stage of our 
knowledge, at this stage of my knowledge, and In the 
exigencies of churning out a Master's thesis. Certainly, 
there are no Inherent restrictions In the capability of 
computers to Incorporate the process. 
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Automatic programming Is the art of 
providing a computer program (an "automatic programming 
system" (APS)) which takes as Input some user-amenable 
description of a task and produces as output computer 
programs to accomplish that task. The user's description of 
his task Is his "model" In the sense described In 1.1.1. 
This Is the "model" which the program described In this 
thesis must debug. 

But why worry about model-debugging? 
Why not let the user specify something, let the system 
generate a solution program, and simply leave It to the user 
to respeclfy the problem If he doesn't like the results? 
There are several answers to this question, some obvious, 
others not so obvious. Basically, the reasons for providing 
sophisticated model -debugging aids revolve around 
considerations of efficient use of the APS, ease of use of 
the APS, ease of construction of the APS, and "safety" In 
the use of the APS. 

The most obvious reason for 
model -debugging Is that since code-generation (I.e., 
actually writing the solution program after the task 
description Is In) Is a rather arduous process, It Is 
worthwhile making sure that the user and the APS agree on 
what the problem Is before the APS actual ly writes programs 
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to solve the problem. This idea of pre-code-generat Ion 
debugging Is as old as compilers, and Is fairly well 
understood. (1) 

A related but not quite so obvious 
reason for providing model-debugging aids In an APS Is to 
make the system easier to use. This Is especially necessary 
In an APS like Protosystem I |9| which attempts to provide 
problem-solving expertise to aid the user. The point Is 
that the APS Is designed to provide problem-solving 
knowledge for a user who Is not at all adept In computer 
problem-solving. To help him design a description of his 
task and then not to aid him In debugging that description 
seems like providing not much help at all: descriptions of 
complex problems "always" have bugs, and finding them Is 
usually as sophisticated a tas"k as writing the description 
In the first place. (2) Thus, I beleve that an APS that 
does not provide model -debugging aid would be difficult, If 
not Impossible, to use. 

Supposing, then, that some kind of 



(1) The actual debugging of models may be quite different 
from the debugging of source code, but the reason for doing 
so Is the same In this case. 

(2) Statistics have shown that about 50% of the time In 
large system development Is spent In debugging |2|. 
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debugging aid Is necessary, how should It be Interfaced with 
the user and with the APS? The answer, I think, Is that 
debugging should occur when the system's knowledge of the 
user's problem Is still at a high level of symbolic 
description. That Is, prior to code generation. This 
leaves the debugging effort In the realm of 
model -debugging. The reason that It Is Important to keep 
debugging at a high symbolic level Is to keep the design of 
the APS as simple as possible. It Is quite difficult to 
maintain the 1 inks between mistakes which occur at low 
levels of description (e.g., programs) and their high-level 
causes. Certainly the user cannot be responsible for 
maintaining these links. If the APS tells him that "an 
Illegal reference was generated from location 11437", we 
cannot expect him to have any notion of what went wrong with 
his model description. In fact, the construction of an APS 
which could make this connection between the bug's 
manifestation and Its cause would be extremely difficult. 
It seems much more reasonable to carry on debugging at a 
high level of symbolic description which both the user and 
the APS can understand In terms of the user's model. 

Finally, there Is a very special problem 
which arises In connection with the use of the APS. The 
user begins to develop a dependency on the APS and to trust 
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the results of the solution programs. When the system fs 
more expert then the user (as Is the case In Protosystem I), 
the user may even trust results v/hlch "common sense" (I.e., 
previous experience, educated guesses, etc.) tells him are 
wrong. In these circumstances, It Is of paramount 
Importance that the user be sure that the APS has a correct 
understanding of his model. Other than the model-debugging 
subsystem within the APS, there may be no source of feedback 
which enables the user to find out that there Is anything 
wrong at all . (1) 

The model -debugging facility has sole 
responsibility for helping the user to understand what Is 
wrong with his model In terms of the model--l.e.. In the 
only terms the user understands. An APS which does not 
provide a facility for Interactive discussion of the model's 
assumptions and their ramifications Is a dangerous tool 
Indeed. Thus, the user must always have some means of 
observing the effects of the assumptions in the model and 
for making sure that the APS "knows what he means". The 
model -debugging subsystem of the APS provides the necessary 
mechanism. 

Therefore, for reasons of efficiency. 



(1) The output code and, In many cases, the assumptions 
underlying Its generation will be Incomprehensible to the 
average user. 
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usability, and safety, a model -debugging facility Is a 
necessary part of an automatic programming system. Still, 
the general problem of model -debugging In automatic 
programming Is much too large to be considered here. In 
the next section, I will explain the particular subdomaln of 
automatic programming I will attack, and my reasons for 
choosing It. 

1.3 stalls, details 

1.3.1 Rp*trlctIon to. £h£. tt£>Ea 

The program described In this thesis Is 
specialized to work on models chosen from the "world of 
business games" (WOBG). By this I mean an environment In 
which the concepts common to business games are the stock 
knowledge. There are several reasons for choosing this 
domain of Interest: (1) the models are sufficiently 
structured to be formally expressible, but are not so 
structured that they are susceptible to mathematical 
analysis; (2) the Interaction of submodels Is the most 
Interesting and complex aspect of the model; (3) this Is one 
of the few domains which Is both reasonable-sized and 
"real-world" (In the sense that there Is a great deal of 
Interest In It Independent of this research); <U) It Is a 
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natural subdomafn of the "world of business" (WOB) of 
Protosystem I I 9 I . 

Models In various domains differ greatly 
In the amount of "structure" present fn the description of 
the model. By "structure" I mean clearly defined rules of 
construction and constraints on elements. The methods used 
In this research require well-defined models. However, If 
the model Is "too wel 1 -defined", debugging becomes 
uninteresting, or Is more easily accomplished by 
mathematical tools. The WOBG seems to have just the right 
level of structure. Since the Idea of modelling business 
systems Is well established, there exist a variety of 
formalisms for expressing business models. These modelling 
formalisms are even more clearly defined for business games. 
The very Idea of a game Is to have a precise set of elements 
and rules for manipulating them. Nonetheless, understanding 
and debugging models of business games is by no means 
trivial. There Is good evidence that users of even the 
simplest of business games have very poor and "buggy" models 
of what Is going on |3|,|6|,|8|. The main reason for this 
Is the complexity of the Interaction between submodels In 
business games. 

I am particularly Interested In 
debugging models In which Interaction of subparts Is a major 
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factor In model complexity. Most model-worlds which have 
been Investigated In artificial Intelligence research (e.g., 
the "blocks world" 1 211) have few complex Interdependences. 
Existing Interaction problems tend to be downplayed In order 
to emphasize other aspects of the models. (For example, 
see Wtnograd's "solution" to the "flndspace problem" In 
|21|; cf. |17|.) I wish to explore the other end of the 
Interdependency scale; I.e., highly Interactive models. (1) 
The kind of models which the program described In this 
research Is designed to debug are those In which the user 
has a good understanding of the various parts of the model, 
but does not understand how these parts (which I will call 
"submodels") Interact v/Ith each other. (2) 

In fact, al 1 of the bugs which the 
program Is designed to find arise from Interaction of 
submodels (see section 3.3). Business games have very 



(1) Real world situations presumably fall somewhere In 
between these two extremes. However, I will devote a 
considerable amount of space (all of section 3) to an 
examination of how Interaction of submodels Is the major 
complexity factor In real world situations (In particular, 
In large business organizations), and how these real world 
Interdependency problems form the "semantic roots" of 
similar problems In the toy-v/orld used In this research. I 
am hoping to motivate an Interest In the "Interaction bugs" 
which will preoccupy the remainder of the thesis. 

(2) I believe that this Is a large and Important class of 
models, Including models of "systems" with wel 1 -understood 
elements (see |3|). 
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precisely defined events (see the example game In Appendix 
A). However, these elements Interact with each other to 
the extent that understanding how the "whole system" (I.e., 
all of the Interacting parts) works Is a major challenge to 
the players. Thus, since poorly understood Interaction of 
submodels Is the major source of bugs In this domain, the 
WOBG forms an excellent testing ground for my program. 

Business games also have the Important 
property of being Interesting In their own right. Playing 
and understanding business games Is considered to be an 
important activity at many schools of management throughout 
the world. There Is therefore little danger of being 
accused of designing a program which works only In an aji bSUL 
problem domain. Furthermore, since people are used to 
trying to model business games for themselves, they can 
appreciate the efforts of a program which aids In the 
debugging of such models. This "real world" flavor of 
business games Is one of their most important properties for 

this research. 

Finally, the WOBG is a natural subdomaln 

of the WOB of Protosystem I. This is useful, first of all, 
because it allows a certain Inheritance of philosophy and 
technique from the larger project. More Importantly, 
though, It enables the model -debugger presented here to be 
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seen In the context of a large automatic programming system. 
Since the ra I son d'etre of my program Is use In an APS / this 
connection with Protosystem I Is an Important aspect of the 

research. 

Therefore / the basic philosophy of 
model -debugging presented here will be applied to models 
chosen from the world of business games. In order to show 
that my basic Ideas about debugging are Indeed "working 
Ideas", I have written a program which uses these concepts 
to debug actual models of business games. 

1.3.2 Ihf: l£l}f! Pf the Program In the thes_L& 

The program presented In this thesis 
serves several purposes: Illustration of Important methods/ 
demonstration of the workability of the techniques, and 
discussion of design Issues for model -debugging programs. 
Certainly, the major use of the program In the thesis Is to 
provide examples for the debugging theory developed In the 
research. All the major debugging Ideas are Illustrated by 
a scenario from the working program. As for the second use 
of the program, a little care Is necessary In explaining the 
"proof" value of the program In the thesis. It Is often 
contended that working programs prove the utility of the 
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theories that they represent. This Is true, so long as the 
reader Is careful not to use some sort of false Induction 
principle to Infer too much from what the program actual 1 v 
jjojg£. As Is almost always the case, the program In this 
thesis can actually do only a subset of what Is talked 
about. I will always make It clear what the program can 
and cannot do, how the program can be extended to do more, 
etc. The reader should draw any general 
conclusions — careful ly--from this Information. 

Using this "program-as- II lustrator" 
philosophy of presentation, I will now launch Into a 
detailed example of program operation on a simple model. 
This will hopefully give the reader a good basic Idea of 
what the rest of the thesis has to say. The Issues raised 
In the example and the example Itself will be discussed at 
length In the rest of the thesis, each aspect of the problem 
appearing In Its logical section (see table of contents). 
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2 jjusi is. give, ^au. siL Idea. . • 

The Important thing to keep In mind 
about this program Is that It finds the causes of bugs In 
much the same way that people (or Sussman's HACKER |18|) do: 
by trying to solve prob1ems--and falling. In this section I 
will present the complete works of my program In connection 
with a very simple example. A lot of new notation Is 
presented here; please don't get bogged down In It. I 
present It here only to avoid vagueness In showing what the 
program actually works with. More complete explanations of 
all the notation (and Indeed, the entire example) appear In 
the appropriate sections later on. This discussion focuses 
on what the program means by a "bug" and on some of the 
procedures used to go from the manifestation to the cause of 
a bug. Neither the procedures nor the descriptive 
mechanisms used by the program are discussed In detail here. 
Philosophical Issues about representation of knowledge In 
the program and goal -prog ramming are eschewed completely. 
This Is a quick "Introduction by doing" to the methodology 
of the program. 

Suppose the user presents the program 
with the following (tiny) model: 
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Consider the following model 
of sales. A sale Is a probabilistic occurence 
which depends only on the amount of advertising 
done. Advertising costs $3000 per page and Is 
good for one quarter. I buy three pages of 
advertising per quarter. If the money to do so Is 
available. Sales take place during sales calls. 
There Is one call per salesman per quarter. A 
customer never buys more than one unit. If a unit 
Is sold, the company records $5000 In accounts 
receivable (A-R), which Is not collectable for 
another two quarters. At any time, a salesman has 
a 5% chance of quitting. If a salesman quits, a 
new man Is hired. After three months of 
training, this man becomes a salesman and may 
start making calls. Both salesmen and trainees 
are paid $1000 per quarter. Trainees also have a 
5% chance of quitting at any time. 



The user would Input this model Into the program with the 
model specification language presented In section k.l. In 
these MSL terms, the model looks like: 

(♦ACTIVITY HIRING 

(♦PREREQUISITES (♦PRESENT (1000 CASH))) 

(♦SCHEDULE ON QUIT) 

(♦PRIORITY 2) 

(♦OUTPUT (SOME TRAINEE)) 

(♦TAKES 0) 
) 

(♦ACTIVITY ADVERTISING 

(♦PREREQUISITES (♦PRESENT (3000 CASH))) 

(♦SCHEDULE 3) 

(♦TAKES 1) 

(♦PRIORITY 3) 

(♦OUTPUT (1 PAGE-OF-ADVERTISING)) 



(♦ACTIVITY TRAINING 

(♦PREREQUISITES 
(AND 



(♦PRESENT (1000 CASH)) 
(♦PRESENT (SOME TRAINEE)) 
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) 

(♦TAKES 3) 

(•OUTPUT (SOME SALESMAN)) 

) 



(♦ACTIVITY SALES-CALL 

(♦PREREQUISITES 
(AND 



(♦PRESENT (1000 CASH)) 
(♦PRESENT (1 UNIT)) 
(♦PRESENT (SOME SALESMAN)) 



) 

) 

(♦TAKES 1) 

) 

(♦ACTIVITY COLLECTION 

(♦PREREQUISITES (♦PRESENT (5000 A-R))) 

(♦TAKES 2) 

(♦OUTPUT (5000 CASH)) 

) 

f*FVFMT SALE 

1 (*CONOITIONS SALES-PROBABILITY) 

(♦ACTIVITIES (SALES-CALL) 

(♦OUTPUT (SOOO A-R)) 

) 
) 

(♦EVENT QUITTING 

( ♦COND I Tl ONS QU I TTWS-PROBAB I L ITY) 
(♦ACTIVITIES (SALES-CALL) 

(♦CANCEL) 

(♦REMOVE (THAT SALESMAN)) 

(♦ACTIVITIES (TRAINING) 
(♦CANCEL) 
(♦REMOVE (THAT TRAINEE)) 

) 
) 

(♦FUNCTION SALES-PROBABILITY „.„^, 

(♦ARGUMENTS (PAGE-OF-ADVERTISING) ) 
(♦RETURN ad- function)) 

) 

(I will not show the exact nature of 
"ad-f unction", as It is a ♦TABLE construct (see U.D — 
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just a bunch of numbers that we shouldn't worry about 
here (see Appendix B).) 



Now suppose the user gives the program 
the following: 

(♦SIMULATE k 1 

((30000 CASH) 

(50 UNIT) 

(DON SALESMAN) 

(MARK SALESMAN) 

(STEVE SALESMAN) 

(BILL SALESMAN) 

(.05 QUITTING-PROBABILITY)) ) 

or, In words, simulate the model for k quarters, showing a 
time-slice every quarter, and with the given Initial values. 
Before considering the actions of the program. It Is 
worthwhile to note a few things. 

First, observe that the the user has 
given the model (50 UNIT) as an Initial resource. This Is a 
typical example of a model-testing technique: adding slack 
to decouple submodels. Presumably, UNIT Is something 
created by another submodel which the user does not wish to 
consider at this time. The user effectively removes this 
"other submodel" by making sure that the submodel Is never 
limited by the amount of UNIT available. (The PRODUCTION 
submodel which creates UNIT's Is shown In Appendix B.)) 

Second, note that we are making an 
Implicit assumption about what the user will do with the 
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simulation after It Is presented by the program. We are 
assuming that he will be either satisfied or dissatisfied 
with the result (1) . If he Is dissatisfied, he will express 
his expectation to the system In the form of a goal. This 
Initiates the debugging process. At this time, let us 
rejoin our example. In progress. 

The first action of the program Is to 
simulate the model as the user requests. If the user's 
expectation Is fulfilled, no further action will be taken 
until the user's next request far simulation. If his 
expectation Is not met, the program will help him find the 
bug In the model . 

The requested simulation Is shown below. 
The representation used here (and throughout the thesis) 
should be seen as a graphical description of the complex of 
list structure which the program uses to describe simulation 
histories. Every part of the diagram has an analog in the 
Connlver 1 20 1 representation of the program (see section 
U.2). 



(1) We are also assuming that the user Is a good judge of 
the overall performance of the system he Is trying to model. 
This is of course not Inconsistent with our basic premise 
that the user does not fully understand the workings of the 
system (and therefore has bugs In his model). Rather, we 
are saying that the user knows pretty well what the model 
should do, but Is having trouble making the model do what it 
should. 
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SIMULATION-HISTORY 



♦TIME-SLICE 0* 
RESOURCES: 



SALESMEN: DON, STEVE, MARK, BILL 
CASH: 30000 

UNITS: 50 



♦TIME-SLICE 1* 



RESOURCES: 



SALESMEN: DON, STEVE, MARK, BILL 
CASH: 17000 

UNITS: U8 

A-R: 10000 



SCHEPULEP ♦activity's: 

SALES-CALL (DON) 
SALES-CALL (STEVE) 
SALES-CALL (MARK) 
SALES-CALL (BILL) 
ADVERTISING 
ADVERTISING 
ADVERTISING 
COLLECTION 
COLLECTION 



♦EVENT'S: 



SALE 
SALE 



(BILL) 
(DON) 



(TIME-LEFT 
(TIME-LEFT 



2) 
2) 



♦TIME-SLICE 2* 
RESOURCES: 



SALESMEN: DON,MARK,BILL 
CASH: 5000 

UNITS: l»7 

A-R: 15000 

TRAINEE: G0001 



SCHEDULED * ACTIVITY's : 

SALES-CALL (DON) 
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•EVENT' s: 



SALES-CALL (MARK) 

SALES-CALL CULL) 

ADVERT I SFNG 

ADVERT IS IMG 

ADVERTISING 

COLLECTION (TIME-LEFT « 

COLLECT ION (TIME -LEFT * 

COLLECT I OH (TIHIf-LEFT - 

HIRING 

TRAINING (TIME-LEFT - 3) 



SALE (MARK) 
QUITTING (STEVEN 



1* 



♦TIME-SLICE 3* 



MARK, BILL 
2000 
46 



RRESOURCES: 

SALESMEN: DON, 

CASH: 

UNITS: 

A-R: 

TRAINEE: GOO 01 



SCHEPUtEP « activity's: 

SALES-CALL (DON) 

SALES-CALL (MARK) 

SALES-CALL (BILL) 

ADVERTISING 

ADVERTISING 

ADVERTISING 

COLLECTION (TIME- LEFT - 2* 

COLLECTION (TIME-LEFT « 1} 

TRAINING (TIME-LEFT » 2) 



'EVENT'S: 



SALE (BILL) 



•TIME-SLICE 4* 
RESOURCES: 



SALESMEN: DON, MARK, BILL 
CASH: 1000 

UNITS: %S 

A-R: 10000 
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TRAINEE: G0001 

SCHEPVIEP » ACTIVITY' s i 

SALES-CALL (DON) 

SALES-CALL (MARK) 

SALES-CALL (BILL) 

ADVERTISING 

COLLECTON (TIME-LEFT » 2) 

COLLECTION (TIME-LEFT = 1) 

TRAINING (TIME-LEFT » 1) 

*EVENT»s: 

SALE (MARK) 



The simulation has resulted fn 5 SALE's. 
Suppose that the user expected 6. There fs a bug fn the 
model --but where? Note that the model runs out of CASH In 
the last quarter (and therefore cannot schedule all three 
ADVERTISING *ACTI VITY's) . However, the bug Is not "NOT 
ENOUGH CASH". Rather, this effect Is symptomatic nf fh» 
bug. Most of the effort of the program Is to point out 
bugs, not their symptoms. But this requires problem-solving 
In the context of the simulation history. Back to the 
actual action of the program... 

The user notes that there were only 5 
SALE's rather than the expected 6. In order to try to 
rectify things, the user gives the system 

(•GOAL (INCREASE SALE D) 

The program Is now In the debugging business. It must try 
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to solve the problem of Increasing the number of SALE's In 
the context of the given simulation history. The places at 
which It encounters dubious constraints In the simulation 
environment are Its possible locations for bugs. 

The program uses the model and the 
simulation history to perform the requisite problem-solving 
activity for each goal as It Is presented. This may be 
thought of as asking two questions of the model and the 
simulation: 

(1) Why didn't you do this before? 
and, If there Is no good reason, 

(2) How could we do this? 

The method of asking and receiving answers to these 
questions Is best explained by continuation of the example. 

The first goal (given by the user) Is 
(*GOAL (INCREASE SALE 1)) 

Since this goal was given by the user, the first question Is 
not asked. However, the second question Is asked. How can 
we Increase the number of SALE's? By examining the model 
and using the logic of INCREASE (explained In section 
k.k.l), we see that one way to Increase SALE's Is to 
Increase the probability of a SALE occur I ng. Thus, the 
system generates a new goal 
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(*GOAL (INCREASE SALES-PROBABILITY)) 

Now the program asks question number one: why wasn't 
SALES-PROBABILITY higher In the first place? The program 
looks at the simulation history and notes that the 
SALES-PROBABILJTY was at a low In tlme-slfce k. Why Is It 
so low? There was not enough ADVERTISING, the program 
determines. This Is a BAD REASON: the model was 
RESOURCE-LIMITED. Okay, how can we get the necessary 
ADVERTISING? In order to Investigate this question, the 
program generates a new goal 

(*GOAL (SCHEDULE 2 ADVERTISING l»)) 

which means "try to schedule 2 ADVERTISING *ACTIVITY's In 
time-slice k n . (The fact that we need 2 ADVERTISING 
♦ACTIVITY'S Is presumably due to the exact nature of 
"ad-f unction", and will not be discussed here.) Again, the 
program asks why the ADVERTISING *ACTIVITY's were not 
scheduled In the first place. The answer Is that there was 
not enough CASH; still RESOURCE-LIMITED, so we pursue this 
1 Ine with: 

(•GOAL (INCREASE CASH 6000 k)) 

By again asking the questions and forming new goals, the 
program forms the following *G0AL line: 

(*G0AL (INCREASE CASH 6000 U)) 
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(*GOAL (SCHEDULE 2 COLLECTION k)> 

(•GOAL (ALLOW 2 SALE 2)) 
(*GOAL (SCHEDULE 3 ADVERTISING 2)) 

("ALLOW" rather than "SCHEDULE" because SALE Is an *EVENT.) 
Note that we are back to SCHEDULlng ADVERTISING. Are we In 
some kind of loop? No, we are moving back In time. 
Furthermore, this time, when we ask why we didn't schedule 
three more ADVERTISING *ACTlV|TY*s In time-slice 2, we find 
that the reason Is that the user told us not to (via his 
♦SCHEDULE specification In the ADVERTISING *ACTIVITY (see 
page 17)). Thus, ADVERTISING Is SCHEDULE-LIMITED In 
time-slice 2. This Is a GOOD REASON, and the program 
terminates action on this line of thought. Nonetheless, It 
saves Information about the terminated line. If no more 
"likely" bug Is found, the program wll 1 tell the user that 
his *SCHEDULE specification for ADVERTISING Is Insufficient 
to allow the model to meet his expectations. In the 
meantime, however, the program explores the model for more 
likely bugs. The program does this by "backing up" (1) some 



(1) This Is not automatic backup In the PLANNER sense. The 
program backs up only In certain cases, and only under 
program control. More Importantly, the effects of the 
"backed-over" *G0AL's are "undone* smlX 1& 1M context S± 
the simulation history . The terminated Vines must be saved 
for later examination by the program. This Is essential for 
handling the *GR0UP constructs discussed later in the 
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and trying a different line of attack. 

In this case, the program checks to see 
If there Is another way to accomplish 

(*G0AL (ALLOW 2 SALE 2)) 

Using Its usual question-asking procedure, the program finds 
the alternate 1 Ine 

(*G0AL (ALLOW 2 SALE 2)) 
(*GOAL (INCREASE SALES-CALL 2 2)) 
(*G0AL (INCREASE SALESMAN 2 2)) 
(*GOAL (SCHEDULE 2 TRAINING -1)) ??? 

(Note that CASH does not have to be INCREASEd In this line 
because there Is already a sufficient amount to support the 
new INCREASES.) The program Immediately notes that It Is 
trying to schedule In negative time, and terminates the 
line. 

This finishes off the entire 
(*GOAL (INCREASE SALES-PROBABILITY)) 

Idea. But there Is still another way for the program to try 
to get that extra SALE It Is looking for: by trying to 
Increase the number of SALES-CALL 1 s. Thus, 

(♦GOAL (INCREASE SALE 1)) 



thesis, and for making final debugging recommendations (see 
section k.k). 
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(*GOAL (INCREASE SALES-CALL 2 It)) 

(•GOAL (INCREASE SALESMAN 2k)) 

(•GOAL (SCHEDULE 2 TRAINING 1)) 

(•GOAL (INCREASE TRAINING 2 1)) 

(•GOAL (INCREASE HIRING 2 I)) 

(The choice of time-slice k for INCREASIng SALES-CALL was 
not arbitrary: the program chooses a slice where It thinks 
It can do the most good.) But the program cannot accomplish 
this last goal. Why not? The user specifically said not to 
hire until someone quits. The program then checks to see 
If HIRING did In fact occur. Yes— one time-slice later. 
This particular set of circumstances suggests a common 
timing bug In the managers M f I re-f Ight Ing" approach to 
problem solving— no action was taken until It was too late 
for It to do any good (the solution Is to anticipate 
problems; more details about managers' bugs In section 3). 
Since this bug arises from so specific a group of events, 
the program thinks It is a rather probable bug and gets 
ready to suggest It first. It then checks to see If there 
are any other ways of INCREASIng the number of SALE's. 
Since there are not. It Is finished looking for bugs, and Is 
now ready to suggest the bugs It knows. 

As advertised, the first bug suggested 
to the user Is: 
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—BAD *SCHEDULE FOR HIRING: DEPENDENT ON QUIT; HIRING 
TOO LATE 



The user may agree that this Is the bug (I think It Is), 
or ask the program to try again. The next bug suggested 

—BAD SENSE OF PRIORITIES: HIRING AND ADVERTISING 

Essentially, the program suggests that It could have 
gotten more ADVERTISING If HIRING did not have higher 
priority. If the user doesn't buy this, the program 
suggests that he simply blew the *SCHEDULE specification 
on ADVERTISING: 

—BAD *SCHEDULE FOR ADVERTISING: NOT ENOUGH 

If the user still doesn't like what's happening (and 
since the program has suggested all of the bugs It 
found), the program decides to see If the user might have 
mls-speclf led or completely omitted a relevant part of 
his model (this happens more often than you might think) 
It then uses Its access to WOBG knowledge to suggest 
—MISSING *ACTIVITY: FACTORING 

(the user may factor accounts-receivable to provide 
Instant cash) and 

--MISSING *ACTIVITY: RESEARCH AND DEVELOPMENT 
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(the user may Increase the probability of a sale by 
Improving his product). 

The program goes out of the debugging 
business whenever the user takes a suggestion, or, of 
course, when Its bag of tricks Is exhausted. The user 
can now fix his model or change his expectations and 
re-slmulate* Eventually, this process of simulation and 
debugging will converge to a model that the user Is 
confident that he and the APS both understand 
sufficiently. 

In this section I have tried to show 
a complete example of what this thesis Is about. I will 
now go Into an examination of the foundations of this 
approach, and the techniques that allow Its 
Implementation. I begin with a philosophical discussion 
of bugs Cyech). 
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3 JLU££ 

A bug Is something that prevents 
something from behaving the way someone expects It to. 
This section particularizes the notion of "bug" to a 
concept which Is useful for this research. As usual, the 
program only knows about a narrowed-down version of 
"bug". 

We will be Interested here only in 
"understanding-bugs"— I.e., bugs that exist only In the 
user's understanding of the system he wishes to model 
(cf. Goldstein's "semantic bugs" |5|). This Immediately 
removes from consideration "parenthesis errors" and other 
"syntactic bugs" (of course, trivial syntax bugs 
sometimes arise from a basic misunderstanding). Thus, 
there will be no Interest whatsoever In finding bugs due 
to MSL errors. In fact, no attention Is given to bugs of 
any kind that arise from careless expression of the 
user's knowledge In the modelling formalism. 

The kinds of bugs with which the 
program Is concerned are those that seem to be "Inherent" 
In the way people understand (or misunderstand) systems. 
The rest of this section will be devoted to an 
examination of bugs that occur In the modelling process 
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and the features of the problem domain that cause them to 
occur. 

3 . l jsjjlels. la model s 

3.1.1 What did 1 dja wrong ? 

What happens when people try to model 
systems? They usually do some mumbling and 
head-scratching and come out with some sort of expression 
of their Ideas. In this research, the "expression" Is 
required to be rather formal, but this doesn't matter 
much. Hext, the modeller somehow tests his model to see 
how It performs under various conditions (just as my 
system uses simulation, see section 4.2). Most of the 
time, the model does not perform as the mode Her expects 
It to — "something goes wrong". 

Actually, "something jasaJL wrong" at 
define-time: there is something In the definition of the 
model which Is causing the unexpected behavior. I have 
already mentioned the hypothesis that the user has a good 
understanding of each submodel. (1) Thus, the part of 
the model definition which Is In error must be a 



(1) The notion of "submodel" will become much more precise 
when I discuss MSL In section 4.1. 



Page 37 

specification of submodel Interaction. The 
manifestation of such a bug varies widely with the 
particular bug Involved, and tends to be a detailed 
matter (I.e., highly dependent on the actual 
representation formalism). Therefore, I will postpone 
(th discussion of this problem until after I have 
described the formalism U.i*.2), and go on to an 
examination of the "semantic roots" of these "Interaction 
bugs". 

5.1.2 Interaction hu&s. 

In order to understand the Idea of 
Interaction between submodels, It Is helpful to view the 
model as a process which defines the action of the 
modelled system. Thus, the models we will examine here 
all "do something". The model can be seen as a system 
which converts some sort of Input resources Into some 
predefined outputs. (This Is, In fact, a very popular 
view of management systems.) For the model to "do" 
anything, Its submodels must Interact with each other. 
That Is, the Inputs to the entire model are actually 
Inputs to certain submodels which convert them Into 
Intermediate quantities which are In turn Inputs to other 
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submodels --and so on until the desired outputs are 
obtained. 

Via this Interaction, various 
dependencies between submodels arise. The most common 
Is that one submodel must waft for the completion of 
another before It can begin action. (See section k.h for 
a detailed account of different kinds of Interaction 
between MSL submodels.) Also, submodels often share 
basic resources, giving rise to conflicts between 
submodels. 

These dependencies and confl tcts 
between submodels provide the environment for the 
following basic "Interaction bugs**: 

(1) Unexpected conflict arising from competition for 
shared resources 

(2) Weak performance due to poor perception of 
time-phased occurences 

(3) Special complexity problems arising from the 
concentration of (1) and (2) In "tight systems" bound 
by higher-order constraints 

Although I believe that these bugs have considerable 
generality, I will not discuss them In the abstract. 
Instead, I will move Immediately Into the domain of 
management systems to provide a framework for discussion. 



3.2 Interaction Jn management systems 
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The bugs catalogued In the above 

subsection arise from poor understanding of complexity. 

This "complexity" Is directly Inherited by the models from 

the modelled domain. As an Introduction to the Interaction 

complexity of organizations In the world of business (which 

form the basis for business games, the "modelled domain" of 

this thesis), I will quote In full an Illustrative passage 

from Galbratth ]k\i 

There Is considerable variation in the 
amount of Interdependence In organizations. The 
kinds of variation can be illustrated by 
considering a large research and development 
laboratory employing some 500 scientists who are 
pursuing the state-of-the-art. Thus we have a 
large number of elements and high task 
uncertainty. However, there Is little need for 
communication. All the projects are small and not 
directly connected to other projects. Therefore a 
schedule delay or a design change does not 
directly affect other design groups. The only 
source of Interdependence Is that the design 
groups share the same pool of resources--men, 
facilities, Ideas, and money. But once the 
Initial resource allocations are made, the only 
necessary communication between design groups Is 
to pass on new Ideas (Allen, 1969). This type of 
interdependence has been 
(Thompson, 1966, Pp. 54-5 ) . 

If the nature of 
Is changed from 250 small Independent 
large ones, a different pattern of Interdependence 
arises. The large projects will require 
sequential designs. That Is, a device Is first 
designed to determine how much power It will 
require. After It Is complete, then the design of 
the power source can take place. Under these 
conditions, a problem encountered In the design of 



termed as 



pooled. 



the projects 
ones to two 
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the device will directly affect the group working 
on the power souce. The greater the number of 
problems, the greater the amount of comunlcatlon 
that must take place to jointly resolve problems. 

The second example describes a 
situation which Is more complex and requires 
greater amounts of Information processing. The 
second example has all the problems that were 
described In the first example. There must be 
budget and facilities allocations made under 
conditions of uncertainty. There must be a flow 
of new Ideas among the technical specialties. 
But, In addition, the second example requires 
Information processing and decision making to 
regulate the schedule of sequential activities. 
This Is because there is greater Interdependence 
In the second example. 

The Interdependence or 
Interrelatedness of the design groups can be 
Increased above what Is described In the second 
example by the degree to which "design 
optimization" Is pursued. Optimization means that 
a highly efficient device Is desired and any 
change In the design of one of the components 
requires redesign of some others. 

This can be Illustrated by an 
automobile engine and body. The handling 
qualities of a car depend on the weight of 
the engine. The engine compartment can hold 
only a certain size of engine with Its 
accessories. The drive shaft and 
differential can handle only a limited 
amount of torque. Changes In the weight, 
size, or output of the engine may 
necessitate changes In the body of the 
automobile. These Interrelations and many 
others must be taken Into account In the 
design of an automobile. 

Actually, In the case of a 
passenger automobile there Is a good deal of 
flexibility with regard to body-engine 
match. The engine compartment Is usually 
large, the parts of the suspension are 
easily changed, and the drive shaft probably 
has plenty of excess torque-carrying 
capabllty. Engines of a variety of shapes 
and sizes are frequently placed In the same 
body. But this need not be the case. In 
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high-performance automobiles, the size of 
the engine compartment Is frequently sharply 
constrained by aerodynamics considerations. 
There may be efforts to lighten the whole 
automobile by making parts of the drive 
system and body as light as possible; given 
the required strengths. In such a 
situation, the flexibility In the size, 
shape and performance of the engine placed 
In the body Is sharply reduced or 
eliminated. (Glennan, 1967) 

Thus the high performance auto Is a highly 
Interrelated system while the passenger car Is a 
flexible, loosely coupled system. The same is 
true of organizational subunlts which must design 
these systems. Any change In the engine design 
for the high performance car must be communicated 
to the group designing the body so that an optimal 
fit is still achieved after the change, This Is 
less true for a passenger car. Therefore, the 
organization designing the high performance car 
must be capable of handling the Information flows 
described in examples one and two for budgets, 
Ideas, and schedules and also those for all 
design-redesign decisions deriving from the 
interrelated design. The amount of Information 
that must be processed increases as the amount of 
Interdependence Increases. 

Each of Galbralth's examples Illustrates 
a kind of Interdependency between subunlts of an 
organization. The first kind, pooled Interdependency . 
gives rise to interaction bug (1) of the previous 
subsection. That Is, when resource sharing is present, there 
is liable to be unexpected conflict between subunlts trying 
to use the same resources (These are the PRIORITY bugs of 
the example In section 2). Galbralth next cites an example 
of sequential interdependency . I.e., interaction over time 
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as well as resources. Again, this second kind of 

Interdependency provides an environment for the second kind 

of Interaction bug: when subunlts Interact over time, the 

modeller Is liable to mis-estimate time-effects, thus 

causing degraded performance (these are the SCHEDULE bugs of 

the example In section 2). Finally, Galbralth mentions 

hifiher-order constraint Interdependent. (1) Essentially, 

this means that a higher-order objective, shared by a group 

of subunlts, has forced a need for greater Interdependency 

among the subunlts of the group. What has happened Is that 

In the new "tighter" system, the pooled and sequential 

Interdependency has been spread to more (sometimes al 1 ) 

members of the Interactive group. This kind of 

Interdependency has a direct Interpretation In the WOBG 

which will be discussed In the next subsection. The third 

kind of Interaction bug from section 3.1.2 of course arises 

from the higher-order constraint environment. (There are no 

examples of this kind of bug In the example of section 2; 

higher-order constraints were deliberately kept out for the 



(1) 

I think that the Introduction of the "design 
optimization" term here Is very unfortunate. The point Is 
that the subunlts have become more Interactive due to the 
presence of a higher-order constraint. In this case, the 
constraint happens to be that the units must Interact In 
order to achieve an optimal design. However, In the next 
subsection I will discuss other higher-order constraints 
which cause ihe same Increase in Interaction. 
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sake of simplicity. There will be examples of this kind of 
bug later In the thesis.) 

These three types of Interdependency 
form the semantic roots of the bugs considered by my 
program. In the fol lowing subsect Ion we wll 1 examine the 
way these real world organizational dependencies are 
modelled In the world of buslnes games. 

3.3 su&s. in ilQfifi madfiis. 

Business games provide a laboratory for 
teaching managerial decision-making. Since most Important 
management decisions Involve resolving conflicts (or 
POSSfblfi Conflicts , In the case of planning) arising from 
subunlt Interdependency, the three kinds of 
Interdependences discussed In the previous section are 
emphasized In many business games. And, of course, with the 
three Interdependences come the three Interaction bugs. 

Pooled Interdependency arises from a 
natural sharing of resources by different parts of the 
game-player's "business". The business game contains a 
very well-defined set of "resources" Cc^sn, salesman . 
Product lon -liHfi&, etc.) which the player must manipulate 
accord ng to certain specified rules of play. (1) The basic 
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Idea ts to accumulate certain resources which are designated 
as "assets". There are a variety of strategies for 
accumulating assets (e.g., use research, do some 
advertising, learn about market trends, etc.). The 
Important point for us Is that the Implementation of anv 
strategy requires manipulation of various subunlts of the 
player's "business". These subunlts share the pooled 
resource of cash . Since cash Is In limited supply, an 
Interdependency Is set up, and conflicts arise. Poor 
understanding of this pooled Interdependency gives rise to 
section 3.1.2's bug type (1): "unexpected conflict arising 
from competition for shared resources." 

A much more Interesting aspect of the 
particular game I have selected Is the sequential 
Interdependency among subunlts. First of all, note that 
some of the activities of the subunlts are "long-term" ^. 
(research and development, training sales personnel, 
constructing additional production capacity, etc.), while 
others are "short-term" (advertising, factoring accounts 
receivable, hiring, etc.). Second, there Is considerable 
linkage between the requirements of some activities and the 



(1) This discussion Is based on the actual business game 
presented In Appendix A — It might be a good Idea to glance 
over the description of the game to give yourself the flavor 
of what's going on. 
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"outputs" of others (production provides units to sell, 
hiring provides employees to train, etc.). Finally, the 
game contains a rather rich "possibility space" for any 
given strategy If the time-scale Is long enough. That Is, 
there are a variety of non-independent ways of going about 
achieving a given task over time. All of this (plus the 
addition of probabilistic occurences over time) adds up to a 
complex pattern of sequential dependecles, which In turn 
gives rise to bug (2), "weak performance due to poor 
perception of time-phased occurences". 

It Is characteristic of the game used 
here (and of most other business games) that the pooled and 
sequential Interdependences are frequently made more 
Intense by "higher-order constraints". These constraints 
arise from the activity structure of the game. The key 
factor Is that various activities and functions of the 
organization depend on the outputs of more than one prior 
activity (note that this was not the case In the example of 
section 2, and thus this problem was avoided). I can 
present a detailed account of these mutual Interdependency 
relationships only after I discuss the way the game Is 
modelled In MSL ( I will do this In fc.U. For now, It will 
suffice to say that two kinds of higher-order constraints 
are distinguished: the kind In which several activities (or, 
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more usually, chains of activities) must combine to provide 
resources for another activity, and the kind In which a 
number of activities can combine In various unstructured 
ways to achieve a functional! y-determlned goal. 

This section has been devoted to filling 
In rather general background Information about the kind of 
bugs the program knows about and how these arise naturally 
In real world systems. We now go on to an examination of 
how the program Incorporates some knowledge about these 
bugs, and how It goes about using this knowledge to debug 
model s. 
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* hsm Xhs. program works 

In this section I will present a program 
which finds the kind of Interaction bugs discussed above. 
An example of program operation has already been shown In 
section 2. From this example/ the following pattern of 
program operation Is evident: the program starts with a 
model represented In a special formal language; It takes 
this model and produces a simulation of It; If the user 
finds a discrepancy between his expectations of model 
performance and the results of the simulation/ he presents 
the program with the goal of eliminating the discrepancy. 
The program then attempts, using both the model as 
originally stated by the user and the results of the model's 
simulation/ to achieve that goal; In the course of falling 
to achieve that goal (1) , the program finds features of the 
model which It considers to be unintended causes of the 
failure — Jmg£. It then suggests these bugs (In order of 
"1 Ikel Ihood") to the user, leaving him to take the next step 
(and perhaps re-lnltlate the process). 

This section considers each aspect of 



(1) The program shou ld fall to achieve almost all user 
goals! (The "almost" Is due to probabilistic 
considerations.) Otherwise, there was not a bug and the 
simulation would have achieved the goal In the first place. 
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this process In turn. It begins U.l) with an examination 
of the model specification language, providing a firm basis 
for understanding what the program does and does not know 
about the user's model. Next (U.2), It describes the 
simulation of the model and the way the results of the 
simulation are presented to the debugger. Continuing along 
the debugging process, section 4.3 deals with the way user 
goals are formed and the way In which the system handles 
goals. Section k.k can then talk about how the program's 
deductive mechanisms pursue goals and locate bugs— the real 
guts of the debugging problem. Finally, there Is a short 
section U.5) on the way the program uses real -world 
knowledge In the debugging process. 

Into the heart of darkness... 

».l Ihft model specification language 

In order for the program to use the 
slmulate-and-Investlgate method for debugging models, the 
models must be represented In a form that Is executahl*. (by 
the simulator) and a form that Is examinable (by the 
problem-solving routines). The model specification 
language (MSL) Is an attempt to combine these two necessary 
forms In a single language (which also purports to be fairly 
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user-oriented!). 

MSL Is a set of simple primitives which 
can be used to describe model s— especially business game 
models (1) . An MSL specification consists of an 
(unordered) collection of the three basic primitives 
♦ACTIVITY, *EVENT, and *FUNCTION. The basic primitives are 
further described by modifying constructs. The model 
manipulates user-defined value/term pairs called "resource 
variables" (e.g. (1000 CASH), (SAM SALESMAN), etc.). An 
example of MSL specifications appear on pages 17-18, and In 
Appendix B. This section contains a* brief description of 
the syntax and semantics of these MSL primitives. 

The basic MSL construct Is the 
♦ACTIVITY. The concept of "activity" used here is precisely 
similar to the usual business sense of the word: a 
well-defined organizational task which processes some 
commodities or Information that Is used by the organization 
(see section 3.1.2; see also the WOB |9| for Its Information 
on activities). An *ACTIVITY also corresponds to a submodel 
(2) —that thing that the user Is supposed to have a good 



(1) No claim Is made for any "completeness" or "sufficiency" 

Shlrh ™ e !L 0f P r ' m,tIves - Th ese are simply constructs 
which can be used to express my game models. 

ipiV??n M V m se ? In * few minutes that *EVENT's and 
♦FUNCTION'S are also submodels. 
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grasp of (see 3. 1.1). The *ACTIVITY specification looks like 

(•ACTIVITY <*ACTIVITY-name> <modlflers>) 
(1) 

As Is usually the case, the modifiers are the most 
Interesting part of the specification. 

One modifier which Is almost always 
present Is the ♦PREREQUISITES specification. This 
construct expresses the necessary Inputs of an ♦ACTIVITY. 

The *PREREQUISITES specification 
contains an arbitrary number of 

(♦PRESENT <resource varlable>) 

forms grouped (Implicitly) by OR or (explicitly) by AND. 
The basic Interpretation Is that the named <resource 
varlable> must be present (2) for the ♦ACTIVITY to be 
Initiated. If there Is an AND specification, then (as one 
would expect) all of the "AND'ed" resource variables must be 
♦PRESENT. Thus, In 



(1) I will use the following notation: "<" and ">" are 
metalinguistic brackets which surround metalinguistic 
statements. Everything else belongs there. 

(2) Clearly, there are the obvious extensions 
"*MAY-BE-PRESENT", "MUST-BE-PRESENT", etc. I have not found 
these concepts necessary to express the models I have used. 
Therefore, they are not Included In the MSI, even though 
their Introduction would be straightforward. 
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(•ACTIVITY SALES-CALL 
(•PREREQUISITES 
(AND 
(♦PRESENT (1000 CASH)) 
(•PRESENT (1 UNIT)) 
(•PRESENT (SOME SALESMAN)) 
) 



) ) 



there must be (1000 CASH), (1 UNIT), and (SOME SALESMAN) for 
SALES-CALL to be Initiated. 

Some further comment Is necessary on the 
quantification mechanism of *PRESENT. The "SOME" In (SOME 
SALESMAN) represents any name of a SALESMAN In the 
model .That Is, 

(♦PRESENT (SOME SALESMAN)) 

will be satisfied with 

(MARK SALESMAN) or 

(DON SALESMAN) or 

(STEVE SALESMAN) 

Numerical quantifications carry an Implicit "at least" 
modifier. That Is, 

(♦PRESENT (1000 CASH)) 
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will be satisfied with 

(10000 CASH) or 

(1000 CASH) 

but not (999 CASH) 

The "at least" modifier may be explicitly stated, or may be 
changed to AT-MOST, as In 

(•PRESENT (1000 CASH) AT-LEAST) 

(♦PRESENT (5 ERRORS) AT-MOST) 

The "outputs" of an *ACTIVITY are 
expressed by the ♦OUTPUT and * REMOVE constructs: 

(♦OUTPUT <resource varlable>) 

(♦REMOVE <resource varlable>) 

which add or delete the named resource variable from the 
model's resources. 

An ♦ACTIVITY construct may be further 
described by: 

(♦TAKES <number>) 

to Indicate that If the ♦ACTIVITY Is Initiated In time-slice 
H, Its outputs do not become available until time-slice 
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n+<numt?er> . The purpose of this Is, of course, to allow 
the modelling of *ACTIVITY's which take an appreciable 
amount of time to be completed. Another Important 
modifier, 

{•PRIORITY <number>> 

allows the user to Indicate preference In allocation of 
resources to •ACTIVITY'S. Thus, If several ♦ACTIVITY'S are 
vying for the same resource, the one with the lowest 
•PRIORITY <number> has first crack at It (1) . 

•SCHEDULE specifications allow the user 
to give explicit scheduling Information to the simulator In 
order to limit the use of an *ACTIVITY. The specifications 
that have been found useful so far are 

(•SCHEDULE <number>) 

to limit the number of times an *ACTIVITY can be scheduled 
In any tlme-sl Ice, 

(•SCHEDULE (ON <*EVENT-name>) ) 
to allow the scheduling of an *ACTIVITY only In the same 



(1) Again, obviously, this simple mechanism could be greatly 
expanded. More complex models would require time-varying 
and other computed •PRIORITY- specifications. These have not 
been Included In MSL. 
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time-slice as the occurence of the named ■♦EVENT, and 

(♦SCHEDULE (EVERY <number>)> 

to limit the scheduling of the *ACTIVITY to time-slice 

<number>,2x<number>,3x<number>,etc. 

The above modifiers, along with the 

user's abll Ity to create resource variables and provide 
arbitrary *ACTIVITY structures, allow enough flexibility to 
express all of the ■•ACTIVITY'S necessary to model the game 

In Appendix A (see the model In Appendix B) . There are, 
however, other kinds of submodels to be considered. 

Another basic construct (I.e., 
submodel -specifier) available to the modeller Is the *EVENT. 
This Is used to express parts of the model which are 
"outside of the system"— beyond the organization's direct 
control. These outside Influences are often modelled as 
probabilistic occurences, so that *EVENT's are usually 
associated with the probabilistic parts of the model. 
•EVENT Is very similar to *ACTIVITY In basic syntax: 

(•EVENT <*EVENT-name> <mod!flers>) 

but the modifiers are somewhat different. 

Instead of the *PREREQOT SITES 
specification, a *CONDITIONS list Is stated: 
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(♦CONDITIONS <boolean express!on>) 

That Is, the simulator expects the body of a *CONDITIONS 
list to evaluate to "true" or "false". Usually, the body 
contains some combination (perhaps related by AND or OR) of 
♦FUNCTION names (1) (see below). The Intent is that the 
♦EVENT may not be Initiated unless the <boolean expresslon> 
evaluates to "true". 

Usually ♦EVENT'S affect particular 
♦ACTIVITY'S. The suscteptlble ♦ACTIVITY'S and the actions to 
be taken by the ♦EVENT are expressed within the ♦EVENT by 
the ♦ACTIVITIES modifier: 

(♦ACTIVITIES (<1 1st of ♦ACTIVITY-names>) <actIons>) 

If an ♦EVENT contains an ♦ACTIVITIES construct. It can be 
Initiated only In a time-slice In which at least one of the 
named ♦ACTIVITY'S Is scheduled. 

One rather unusual <action> which can be 
taken by an ♦EVENT Is 



(1) These ♦FUNCTION'S usually express a probability with 
which the ♦EVENT occurs In a given time-slice. The 
simulator sets up a probabilistic event (no confusion, 
please!) on the related sample space to express the 
♦FUNCTION. It then calls a random number generator. If the 
value returned by the RNG falls within the defined event, the 
simulator assigns "true" to the value of that ♦FUNCTION. 
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(♦CANCEL) 

This means that the Interrupted *ACTIVITY has been 
permanently disrupted, and Is to be unscheduled. (Of 
course, It can be rescheduled later.) In all other 
respects, ♦EVENT'S are treated just like ♦ACTIVITY'S. 

The final basic construct In MSL Is 
♦FUNCTION. It expresses a functional relationship between 
variables In the model, and. In general, accounts for 
Information flow within the model. It Is thus slightly 
different In spirit from the resource-handling ♦ACTIVITY'S 
and ♦EVENT'S. Nonetheless, It shares submodel status (1) , 
and Is similar In syntax to the other two basic constructs: 

(♦FUNCTION <^FUNCTION-name> <modlflers>) 

♦FUNCTION'S are not "scheduled"; rather, they are Invoked by 
being mentioned In other constructs (just as In programming 
language function calls). Thus, whenever SALES-PROBABILITY 
(see section 2) appears In the model (except In the 
♦FUNCTION definition, of course), the ♦FUNCTION 



(1) It Is Important to recognize that Information-handling 
activities are submodels at the same level as other 
organizational activities. Forrester stresses this point 
In |3|, and seems to use the homogeneity of basic submodels 
successfully. Of course, the uniform submodel constructs 
also lead to a gain In modelling efficiency and a lessening 
of the cognitive load of the MSL user. 
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SALES-PROBABILITY will be Invoked. 

The analogous construct to 
♦PREREQUISITES and *CONDITIONS In ♦FUNCTION Is 

(♦ARGUMENTS <argumentl> <argument2> ...) 

which behaves like the usual argument-list In programming 
language functions. Missing arguments cause an "error" 
which stops the simulation (1) . 

The analogy to *0UTPUT Is 

(♦RETURN <expressIon>) 

where <expresston> can be a combination of ♦FUNCTION names 
and the special function-representing constructs 

(♦TABLE (<^ARGUMENT-name> ORESULT-name>) 
<argument/ result pa!rs>) 

(♦SUM-UP (<varlable range>) <1 Inear factors>) 

This Is about all there Is to the MSL. 
The semantics of ♦ACTIVITY'S and ♦EVENT's are developed a 
bit further In the next section. ♦FUNCTION'S are dealt with 
In if. k. 2.1. However, no really detailed descriptions are 
presented anywhere. There Is little point In It. The only 



(1) This Is, of course, the kind of bug we're no_L Interested 
in here. 
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purpose of presenting MSL Is to allow the reader to 
understand the examples and judge what the program does and 
does not know about a particular model. 

Almost all of what the program knows 
about any given model Is In the MSL specification. (It 
knows a few other things discussed In «t.5.) MSL can be 
simple because the models considered are quite simple. As 
the models become more complex we expect (by conservation of 
complexity) that MSL will become more complex. The hope Is 
that MSL contains something general enough to handle some 
kinds of additional model complexity without additional 
language complexity. This "something* Is the basic 
philosophy of submodel structuring which Is reflected In the 
MSL. Thus, I have tried to emphasize this basic structure 
rather the details. In the next section we follow the 
course of the program's debugging process and examine the 
simulation of MSL models. 

^.2 Simulation as. a way. &£ doing things 

Simulation Is a technique for observing 
the behavior of models. In the absence of analytical and 
other "high-level" tools (like educated guesses) , simulation 
Is the only way to find out wha a model "does" In any given 
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situation. In the model -debugging system presented In this 
thesis, the simulator sets up the basic feedback mechanism 
between user and APS. 

At the very least, any APS should 
provide a facility for checking out model behavior with 
simulation. That Is, the user formulates his model, tests 
It via simulation, changes it If he doesn't like what he 
sees, and reslmulates. For reasons discussed in the 
Introductory section, It is necessary to go a step further. 
The program described here attempts to aid the user in 
discovering why the model does not perform as he expects it 
to . 

Therefore, this section will concentrate 
on simulation as a way of initiating the debugging process. 
This emphasis Ignores very important issues of presenting 
simulation results to the user. In fact, It completely 
downplays the Importance of the simulator Itself, 
concentrating only on the interaction of the simulator and 
the deductive mechanisms of the debugging program. Thus, 
In this section I will proceed to finesse the simulator and 
move on to the more relevant problems of representing the 
knowledge gained by the simulation In such a way that it can 
be used by the debugger. 
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U.2.1 Ihfi. simulator finessed 

In this section I will very briefly 
describe the simulation scheme used In the program. The 
who 1 e si mul a 1 1 on ph 1 1 osoph y pre sen ted he re is kind of 
strange as viewed from the standpoint of "normal" simulation 
programs. This Is due to the presence of two major design 
criteria not usually found In the area of simulation 
programming: 

(1) Adherence to the "user only knows local submodel 
Information" canon annunciated earl ler (sections 1.3.1 
and 3.1.1) 

(2) The goal of representing knowledge found by the 
simulation In such a way that It can be used by the 
debugger 

The first criterion gives rise to those funny MSI constructs 
which mysteriously appeared In the previous discussion. 
It also motivates the style of simulation described In the 
rest of this section. The second criterion determines the 
actual Implementation of the algorithm, and Is dealt with In 
the following subsection. 

In MSL, the Information pertaining to a 
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particular submodel Is found only In that submodel. The 
kind of "Information" varies from submodel to submodel (as 
described In i».l), but basically, the following 
specifications are necessary: 

--resources needed by the submodel 



--resources produced by the submodel , and the length of 
time necessary to produce them 

--expl Iclt pol Icy for the conditions under which the 
submodel should be activated 

The basic operation of the simulator Is 

then straightforward. Each submodel Is activated when Its 

(user-specified) explicit pre-conditions are satisfied, 

provided that all of Its necessary resources are available. 

If the user does not specify pre-conditions (via *SCHEDULE 

and *CONDITIONS— see «».l), the submodel Is activated 

Whenever Its necessary resources are available (subject to 

•PRIORITY restrictions, of course). When the time 

(specified by *TAKES) for submodel activity has elapsed, the 

output resources of the submodel (If any) become available 

to the whole model. This process of cycling through 

submodels activating "ready" ones, continuing "running" 
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ones, cleaning up finished ones, and augmenting and 
depleting resources all along continues for the duration of 
the user-speclf fed run-length. 

Now anyone who has ever glanced at the 
guts of a simulator knows that I have just finessed 
Inumerable details (as well as a few Important Points). The 
algorithm used In the program Is actually a bit more 
sophisticated and a great deal hairier than the one 
"described" above. For example, I have not even mentioned 
the rather ticklish problem of handling probabilistic 
occurences In this context, nor the design decisions for 
priority-scheduling of already-running submodels. I am 
deliberately slufflng the details here because the simulator 
Itself Is not very Important to the thesis as a whole. It 
Is Its output, the SIMULATION-HISTORY context, that I wish 
to emphasize here. 



u.2.2 Simulation history anA simulation -history 

The form of the output of a simulation 
program Is always a key factor In Its usefulness. In the 
debugging system presented here. It Is an essential link 
between the model and the deductive mechanisms of the 
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debugger. As discussed above, much of the task of the 
simulator Is to present the knowledge gained by simulating 
the model In a form that can be used by the rest of the 
program. This Is of course the old artificial Intelligence 
task of representing knowledge In a form that can be used by 
procedural deductive mechanisms. 

The style of representation I have 
chosen for the simulation knowledge Is the simulation 
history . Now this Is hardly startling — simulation 
histories are frequently used to describe the behavior of 
systems. But here I wish to extend the concept somewhat. 
In my program, the simulator constructs a simulation history 
(called SIMULATION-HISTORY) which then becomes the 
PrPblttffrSPlVlnft environment of the debugger. By this I 
mean that from the point of view of the deductive mechanisms 
in the debugger, the "world" Is a simulation history; I.e., 
a sequence of facts about the model which are true at 
various times determined by the simulation. The debugger 
lives Inside this simulation history. The things that It 
knows about the "world" — the kinds of knowledge found, the 
way events are related, etc.-- are the facts and rules of 
the simulation history world (1) . In thinking about the 



(1) Except for, as we shall see later, the facts It knows 
about the "real world" of business games. 
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debugger, It Is well to keep In mind that It Is a citizen of 
the simulation history world. 

Well then, let's go slumming and look 
around the simulation history world ourselves for a few 
roll Icklng moments. Consider some set of observational 
variables on a simulation model. Then a simulation history 
can be thought of as a recording of the "values" of these 
variables at various Instants of slmulat Ion-time. The 
Interesting questions are what observational variables 
should be used and how the record should be organized. We 
will see that these questions are Important with respect to 
the usefulness of the simulator to the debugger. 

For the simulation to progress from one 
time Instant to the next, the simulator must have a record 
of the iiats. of the simulation. The simulation state of 
these simple MSL models consists of four main pieces of 
Information: 

(1) the value of each "resource variable" (see k.l) at 
the end of each time-slice (1) 



(2) a record of the *ACTIVITY's which were Initiated In 
the tlme-sl Ice 



(1) A time-slice Is one ker-chunk of the simulator. 
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(3) a record of the *EVENT's which occur and the 
♦ACTIVITY'S they affect 

(U) an Indication of the stage of completion of each 
"running" (I.e., previously Initiated and not yet 
complete) *ACTIV|TY and *EVENT 

Therefore, the simulator needs these four pieces of 
Information at the end of each time-slice In order to go on 
to the next time-slice. 



But what does this have to do with the 
"observational variables" for the simulation history? First, 
remember that the "observer" In this case Is the deductive 
mechanism of the debugger. Now, harking back to all that 
was said In sections 1 and 2 about debugging by 
problem-solving, we can see that the debugger Is usually In 
the position of trying to change the course of the 
simulation In some way (to cause some desired outcome which 
causes another desired outcome, etc... which eventually 
causes the user's desired outcome). In order to decide 
whether It can make the change (1) It must know something 



(1) Of course. It must also decide whether the user wants 
the change to be made. This part of the problem Is 
discussed In k.k.2. 
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about the simulation. Specifically, It must know the state 
of the simulation and ways to change that state (1) . The 
ways to change the state are encoded In procedural deductive 
mechanisms to be described later U.*.l). The state of the 
simulation can be provided by the simulation history. 
Therefore, the observational variables for the simulation 
history are just the state variables discussed above (2) . 

Well, since the simulator needs the 
values of the state variables at the end of each time-slice, 
the program need only keep track of these values In some 
useful fashion. The problem now becomes one of organizing 
the simulation history. In order ot think about such an 
organization, we can look back to section 2 and remember a 
bit more about what the deductive mechanisms do with the 
simulation history. 

The deductive mechanisms usually find 
themselves playing around In their little simulation history 
world In two ways: 

(1) examining a single time-slice to see whether a 
change can be made at that time 



iiiJ hlS ,S ?tS " WOr1d know1e <»ee tt of the simulation history 



world. 



(2) A schematic representation of these state variables as 
they appear In the simulation history Is found on pp. 21-23. 
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(2) examining a large segment of the simulation to 
choose a likely time-slfce for scheduling something 
new, to follow the course of an *ACTIVITY or ♦EVENT, to 
pursue the consequences of a proposed change, or (as we 
shall see later In this section) to handle higher-order 
constraints 

What we need Is a good representation for facile handling of 
time-slices and (usually contiguous) groups of time-slices. 
The representation should also allow ease In the bulldlng-up 
and manipulation of the whole history. 

Such a representation Is the Connlver 
context 1 20 1 . The simulation history Is Implemented as a 
Connlver context with the unlikely moniker of 
SIMULATION-HISTORY. Each time-slice Is a laver |20| of the 
context. This Connlver Implementation Implies the following 
relation between time-slices: the simulator "grows" 
SIMULATION-HISTORY by adding on new time-slices; changes 
made to the data In a new time-slice are Invisible to 
earlier time-slices, however, the status of any datum can be 
determined In any time-slice. This certainly gives us the 
record of the simulation history that we want. Connlver 
also allows any part of the context to be regarded as a 
separate context. The Importance of this Is that the 
context can then be used as the database, or, more 
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precisely, as the working environment/ for some set of 
programs. That Is, the programs In a given context work 
only with that context as a knowledge base. Thus, we can 
see that the deductive mechanisms of the debugger can "live 
Inside" the simulation history by simply using 
SIMULATION-HISTORY as their context . Furthermore, the 
deductive mechanisms can live Inside any part of the 
simulation history which they must examine. Their world can 
be a single tlme-sl Ice or a large, program-edited piece of 
the history. 

We will see that this ability to live 
Inside arbitrary pieces of SIMULATION-HISTORY Is a key 
requlstlte for the deductive mechanisms of the debugger. 
For the deductive mechanisms to work, they must apply their 
procedural ly-embedded knowledge of how to change the course 
of the simulation to carefully chosen parts of the 
simulation. This Is why the simulation history and Its 
Implementation as SIMULATION-HISTORY form such an Important 
part of the program. In the next section, we will find that 
the SIMULATION-HISTORY representation gains further 
Importance when the debugger generates hypothetical states 
of the simulation. 

U.3 Goals and environments 
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Throughout the thesis I have been using 
the word "goal" to describe a variety of phenomena. I have 
spoken of user goals, system goals, and submodel goals. In 
section 2 ! Introduced another construct containing the word 
"goal": 

(♦GOAL <strange words> <numbers> <lots of parentheses)) 

which purported to represent the various other kinds of 
goals to the program. In this section I will discuss what 
these parenthetical thlngees mean to the program. In the 
next section I will talk about how they are created and 
manipulated. Here I describe only goals qua *GOAL's-- i .e . , 
the common structural aspects of *G0AL's. 

A goal expresses a desired state. In a 
debugging context this desired state is almost always 
inconsistent with the actual state. This Is because the 
user has found a discrepancy between reality and expectation 
and has thought of a desired state In which the discrepancy 
Is resolved. Thus, the desired state, reflecting the fixed 
discrepancy, Is inconsistent with the actual state. In the 
program presented here, the user can ask the program to 
produce this desired state (given the model and the 
simulation history— see section 2). (1) The request is made 



(1) As discussed elsewhere, the program falls In Its attempt 
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vfa a *GOAL statement: 

(♦GOAL <achleve desired state>) 

What does It mean to "achieve the 
desired state**? The user Is asking the program to change 
the course of the simulation. The program goes about this 
by first creating a hypothetical simulation state 
(time-slice) which Includes the desired state. Then It 
attempts to make the rest of the simulation history (I.e., 
the previous time-slices) consistent with the new 
hypothetical time-slice. (1) This is done by the creation 
of a new ♦GOAL 

(♦GOAL <make previous time-slice consistent with new one>) 

This new ♦GOAL Is clearly of the form 

(♦GOAL <achleve desired state>) 

and can thus be handled exactly like the user goal. The 
program can thus recurse merrily along until It cannot 
achieve a desired state— I.e., until It falls. 

Now then, let's take a closer look at 



to produce the desired state, but this Is not Important to 
the discussion of this section. 

(1) This "work backwards" methodology Is due to the 
debugging philosophy of tracing a bug from Its manifestation 
J2&CJ& to Its cause. 
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this process. Each *GOAL requests a specific change to a 
specific local environment (the time-slice). Thus, each 
♦GOAL Is attempted In the context of a local constraint 
environment represented by a single time-slice of the 
simulation history. (1) If the *G0AL Is achieved, It will 
define a new environment which Is Inconsistent with the old 
time-slice (because of the changes wrought by achieving the 
♦GOAL). This new environment Is then consistent with the 
user's desired state, but Inconsistent with the old 
simulation history. The program will then use this new 
local environment as a basis for defining the next desired 
state along the line toward making the whole simulation 
history consistent with the user's desired state. The 
program Is, In effect, constructing a new hypothetical 
simulation history which results In the user's desired 
state. (2) 

Thus, environments are Intimately 
related to the semantics of *GOAL's. Each *G0AL Is 
constrained by a pre-spec If led part of the simulation 



(1) Not quite. As we shall see in a second, multiple goals 
are achieved with respect to a local constraint environment 
consisting of several time-slices. 

(2) The next section deals with the problem of how the 
program constructs this simulation without destroying the 
original Intent of the model. Specifically, section 4.4.2.1 
gives a better picture of what Is "constraining" about a 

local constraint environment". 
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environment — that part which It Is supposed to change. The 
achievement of a *GOAL can therefore be seen as a 
transformation: 




*GOAL 



\D\hc\ envrircwAWif 




new v/\\)\for\M2yfi 



This transformation Is a local phenomenon. However the 
effects of the transformation are non-local. The *GOAL has 
perturbed the local environment and made It inconsistent 
with the global environment. Since the eventual goal of the 
problem solver Is to create a consistent simulation history 
which results In the user's desired state, the global 
environment must be made consistent with this new 
Inconsistent piece: 




*G0AL 



initial a\\)\fwWfcMt 
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In order to make the global environment 
consistent, the program must trace down the effects of 
changing that local piece. In other words, it must examine 
the way that local piece Interacts with other pieces of the 
global environment: 



*QOf\L' 





*&fe in one, li/ie shks 

But this is exactly what we want. The user is incapable of 
following the interactions of the model. If the program Is 
to help the user find the "Interaction bugs 1 ' thus created, 
it must have some mechanism for tracing Interactions. This 
mechanism is the problem-solver. 

The problem-solver uses a *GOAL to 
express a global environment perturbation. It then uses the 
deductive mechanisms described in the next section to follow 
that perturbation throughout the local environment, the 
local change at each point being determined by a *GOAL. 
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When the program comes to a point where the perturbation 
cannot be continued (I.e., where a *GOAL fails). It has, In 
effect, discovered a part of the environment which cannot h* 
E&& la conform to the user's desired environment. it has 
traced the Interaction path to Its roots-It has bracken 
the bug location between the user's desired simulation state 
and the user's desired constraint which gave rise to the 
Interaction (see k.k.3). 

Thus, *GOAL's are the vehicle for 
exploring the Interactive behavior of the model. As we have 
seen above, the use of *GOAL's In this way requires 
sophisticated manipulations of local environments. In 
order to tie down some of the concepts discussed In the 
Previous paragraphs, I wll 1 now discuss some of the problems 
the program faces with respect to this environment-handling. 

First, each *GOAL must be achieved with 
respect to a local environment. That Is, the *GOAL must 
only "see" the constraints of a local environment (not the 
whole thing) (1) , and must directly affect only that local 
environment. Otherwise, the distinction between local and 
Interactive behavior Is lost-there Is no such thing as a 



pJoc«I---'; e ? Ue uo I I S f«^i the ? atUre ° f the Problem-«olvln, 
fli:; J . set up a local environment and then make the nex 
local environment up the line consistent with lt»--*n 
second to the debusslne ohlln^nhw LIL"!^,", [.. J! an< 



'g 
;t 

'bugging philosophy espoused~In"i"i.2.i. 
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"perturbation". 

Fortunately, the environment to be 
examined Is the SIMULATION-HISTORY context . We will see in 
^.it.2.1 that the required local environment is (usually) 
just a TIME-SLICE of the SIMULATION-HISTORY. The *GOAL can 
thus be made to "see" only a local environment by making the 
required TIME-SLICE Its working environment (as in k.2) (1) 
The context structure makes the relation between 
TIME-SLICE's evident (I.e., because each Is a Connlver 
1aver >» so that the distinction between local and 
interactive constraints Is explicit In the built-in 
(Connlver) semantics of SIMULATION-HISTORY. 

Now the *GOAL must also be made to 
affect onlv a local environment If the semantics discussed 
earlier are to be preserved. It would seem that this is 
just as easy: simply keep the TIME-SLICE In question as the 
♦GOAL's working environment, and all changes will explicitly 
have the required locality. However, there is a 
complicating factor found in all searching problem-solvers: 
the problem-solver must make provisions for discarding an 
old line of attack and beginning a new one. This Is the old 
problem of backup which has been discussed extensively in 



(1) This Isn't quite so simple for multiple *GOAL's, as 
we'll see in a second. 
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I7| and |19|. 

The backup problem Is germane to the 
debugging process because the debugger usually attempts to 
find all possible causes of a particular discrepancy (In the 
hope that one of them Is the actual bug). Thus, It will 
follow down one line of -attack, fall, and try another. It 
must therefore be ready to erase the consequences of the 
line to be discarded. But this Is a particularly hard 
problem for the debugger. Here, the tracks leading to 
failures are the key to the rest of the process. They 
cannot be simple "erased", but must be preserved In some 
form which the program can use to suggest bugs and to 
explain Its actions to the user see k.k.3). 

Furthermore, while the effects of each 
♦GOAL must be restlcted to a local envlronmet, the effects 
of All the ♦GOAL's must create a new consistent environment 
(1) . Thus, the program must maintain some new environment 
which localizes the effects of the ♦GOAL's, allows a 
controlled backup with preservation of the backed-over 
Information, and which forces consistency of all affected 
environments. Certainly, SIMULATION-HISTORY will not do. 

But something like It will. The program 
again uses a lave red-context structure. In each laver It 



(1) They must, In fact, create a new simulation history. 
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records the changes made by a *GOAL to the particular 
TIME-SLICE Involved. It then appends this new laver to 
SIMULATION-HISTORY and uses this new augmented context as 
the working environment of the debugger. Now, remembering 
the little discussion of context semantics In k.2 (or, 
referring to |20|), we see that this causes the following 
effects: 

(1) The effects of a *G0AL are certainly localized 
since they occur only In a single laver which 
corresponds to a single TIME-SLICE. 

(2) The debugger can always see a consistent 
environment by looking up the augmented 
SIMULATION-HISTORY as far as the last affected 
TIME-SLICE; the semantics of context then say that 
the data seen by the debugger Is just what was in 
SIMULATION-HISTORY before (which Is consistent via the 
simulator) except where contradicted by the parts that 
were changed by *G0AL's (which are consistent (up to 
that point) via the deductive mechanisms). 

Perhaps It Is well to Interrupt here with an explanatory 
diagram. . . 
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which fs, due to the semantics of context , equivalent to: 
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which Is certainly an easier conceptualization of what has 
gone on so far. However, the first picture Is necessary to 
explain 
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(3) The layers which record the changes made by a *G0AL 
(the dashed parts of the first picture) can be peeled 
off and saved at any time, thus restoring the context 
to Its original condition and saving the effects of the 
♦GOAL (the track toward falure) for further use 

This methodology fills the bill so far. Unfortunately, 
there Is one final problem which complicates this little 
picture (you just knew there would be). 

This complication comes from an as yet 
unseen aspect of the problem-solver: multiple goals. I 
mentioned earlier (section 3) the existence of "higher-order 
constraint Interdependences" In the model. (This 
weird-sounding effect was conveniently kept out of the 
example In section 2.) We will see In section h.k.2.3 that 
higher-order Interdependency leads to multiple goals. That 
Is, Instead of simple goals, the program must deal with 
constructs 1 Ike: 

(•GOAL (♦AND 

(*G0AL ...) 
(*G0AL ...) 
(♦GOAL ...))) 

and 
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(♦GOAL (*GROUP 

(*GOAL ...) 

(♦GOAL ...) 

(♦GOAL ...))) 

We'll see more about multiple goals later. For now we need 
only examine one aspect of their behavior. 

The raison d'etre of ♦AND and ♦GROUP is 
the expression of the fact that their component ♦GOAL's are 
not independent. That Is, the ♦GOAL's they contain share 
common resources and cannot be achieved at each other's 
expense. (This is how they model interdependency . ) Thus, 
the notion of a "local constraint environment" varies from 
the one bandied about earl ier. Here we must have several 
♦GOAL's sharing a single local environment. Furthermore, 
because of the Interdependence of the ♦GOAL's, a component 
♦GOAL that has not yet been completed must "see" the 
constraints posed by the completion of other component 
♦GOAL's. Thus, the local constraint environment might 
cover several TIME-SLICE's. 

Clearly this hairs things up a bit. 
Nonetheless, the program must preserve the semantics of 
these constructs because they are Important effects of the 
model which give rise to their own special bugs (see 
4. if. 2. 3). Actually, given the flexibility of contexts . the 



Page 82 



Implementation Is rather straightforward. The llttl, 
schematic of environments now looks like: 
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In terms of the previous discussion of 
perturbat Ions, local and global environments/ etc., nothing 
has changed except that the "local" environments now may 
have a hairy mlcrostructure of local environments: 







cWriftq , 



Uninterested readers may squint at the above picture (and 
concept)/ leaving everything as before. 

Thus, a *GOAL Indicates a local 
perturbation. The deductive mechanisms of the 
problem-solver follow through the Interactions defined by 
the model to carry the perturbation throughout the 
simulation history in order to produce a consistent 
environment. The next section considers these deductive 
mechanisms and their interaction (via failure) with the 
bug-finders. 



k . k Debugging £v. probl em-SPl ViPS. 



Page 81* 

The basic task of the program Is to 
trace a bug from Its manifestation to Its source. This Is 
done by taking In the manifestation as a *GOAL to be 
achieved (as discussed earlier). The process of achieving 
such a *GOAL Is usually called "problem-solving". But this 
Is a rather special use of problem-solving: the program 
expects to fall In the attempt. In fact, It Is not until 
after a line of attack has failed that It becomes 
Interesting to the debugger. In this section we see how 
lines of attack are formed, how they fall, and how they are 
used after they fall. 

The most Important part of any 
problem-solving process Is the formation of subgoals (1) 
Section l».i*.l considers the methods (those deductive 
mechanisms we've heard so much about) for devising new 
subgoals In order to achieve a goal. This corresponds to 
asking the "how could we do this ?" question of section 2. 
But In this program, the object of the problem-solver is not 
this direct attack on the problem. Instead, the 
problem-solver must make certain It does not change the 
Intent of the user's model In trying to debug It. 

Thus, the process of attacking the 



(1) Especially In this problem-solver. Since subgoals are 
rarely achieved, the whole process turns Into 
subgoal -format Ion . 



Page 85 

user's goal leads directly into the problem of separating 
the constraints which are In the simulation history because 
of user Intent from those which are artifacts of unintended 
model operation. At certain key points In the deduction 
process/ the program determines whether or not It should (In 
terms of user Intentions) make the changes required by the 
deduction. This process of assigning GOOD and BAD REASON'S 
to model action corresponds to asking the "why didn't you do 
this before?" question of section 2. In k.k.2 we examine 
this REASONing process In terms of the philosophy of bugs 
presented In section 3. 

The REASONing process leaves the program 
with a failed line of attack. This appears as a stream of 
♦GOAL'S/ annotated at each point with the BAD REASON that 
triggered further program action. The program must then 
examine the record of the problem-solver to attach blame to 
the proper offending model part; I.e./ to find the bug. 
This task of post-mortem recrimination Is the subject of 
h.k.3. 

k.h.l Ihs. attack 

Here we examine the problem-solving 
phase of the debugging process. The key problem-solving 
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task of the program Is to find the proper local changes 
throughout the global environment which will lead to the 
desired change. Since each desired change Is represented by 
a *GOAL, the problem-solver proceeds by subgoal formation. 

The subgoal -format Ion parts of the 
program (the "deductive mechanisms" mentioned earlier) are 
responsible for figuring out how one local change can be 
brought about by another. As an example of the way this 
cause-effect knowledge Is procedurally represented In the 
problem-solver, the INCREASE function Is presented here. 
The explanation of how INCREASE works will lead us directly 
Into the REASON I ng methods of U . I* . 2 . 

The program's main vehicle for asking 
the "how?" question Is the INCREASE *GOAL: 

(♦GOAL (INCREASE <resouree variable or submodel> 
<amount> <tlme-sllce> (1) )) 

That Is, "goal: Increase the resource variable or submodel 
by the specified amount In the specified time-slice." The 
user's Initial *GOAL Is usually of the INCREASE type (see 
section 2). This just means that the user's discrepancy Is 
usually a defflclency of some resource variable (or lack of 



ULillt a < tlme " S,Ice> ls not «'ven, the program 
heurlstlcal ly chooses one. 
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the appearance of some submodel) which he is asking the 
program to fix up. 

As we saw In section h.Z, the program 
Immediately sets up a hypothetical local environment In 
which the defflclency has been rectified. Then It tries to 
deduce an earlier environment which would cause the new 
desired simulation state. It does this deduction via the 
"logic of INCREASE" mentioned In section 2. The "logic", 
briefly stated, runs as follows: 

(1) Constant quantities cannot be INCREASE'd 

(2) In order to INCREASE a quantity that Is a resource 
variable which Is *0UTPUT (*REMOVE'd) by an *ACTIVITY 
or *EVENT, set up a new *GOAL to INCREASE (DECREASE) 
the number of occurences of that *ACTIVITY or *EVENT 

(3) In order to INCREASE a quantity that Is *RETURN'ed 
by a ♦FUNCTION, set up a new INCREASE-FUNCTION *GOAL (1) 

(U) In order to INCREASE the number of occurences of an 
♦ACTIVITY, set up (If necessary (2) ) a new *GOAL to 



(1) INCREASE-FUNCTION's major claim to fame Is that It sets 
up *GROUP *GOAL's. I will therefore discuss It when I talk 
about *GROUP In U.if.2.3 rather than here. For now It's okay 
to view INCREASE-FUNCTION as analogous to INCREASE applied 
to *ACTIVITY's. 
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INCREASE the resources needed by that *ACTIVITY 

(5) In order to INCREASE the number of occurences of an 
•EVENT, set up a new *GOAL to INCREASE the frequency 
with which Its *CONDITIONS are valid (which might 
Include a *GOAL to INCREASE the number of occurences of 
the *ACTIVITY's which the *EVENT affects) 

Clearly, the Intent of this list Is to cover anything which 
the user or another part of the program (1) might ask to 
INCREASE. However, the rules In the list are by no means of 
uniform character; they differ greatly In their logical 
bases. 

The first rule can be viewed as a 
"fact", or, If you will, a property of the concept 
"Increase." That Is, the first rule depends only on the 
concept of "Increase" — not on MSL, models, etc. The second 
rule expresses a definite property of MSL rooted In the 
semantics of *OUTPUT. It therefore depends not only on 
"Increase", but also on the definition of MSL. The third 
rule, which will be discussed later, depends on "Increase", 
the definition of MSL, and the rules of mathematics (since 



(2) Some necessary resources may already be present In 
sufficient quantity. 

(1) Since INCREASE Is defined recursively, the "other part 
of the program" might be INCREASE Itself. 
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mathematical functions are being Increased). Again, It Is 
valid for any MSL model. The fourth and fifth rules are 
different In a very Important way. They depend not only on 
the definition of MSL and other "givens", but also on the 
particular model defined by the user. 

The reason for this is that the 
occurence of *ACTIVITY's (and thus *EVENT's via the 
♦ACTIVITIES construct (see i».l>> can be directly determined 
by user Intentions. These Intentions are expressed by the 
♦SCHEDULE modifier (see k.l). *SCHEDULE Is used whenever 
the modeller wishes to override the "always schedule when 
possible" default of the simulator. It therefore determines 
the pattern of *ACTIVITY and *EVENT activation throughout 
the simulation. *SCHEDULE is thus the primary expression of 
the user's policy for directing the dynamics of his model. 

The fact that the "logic of INCREASE" 
must take Into account user intention provides the key link 
between the "how?" and "why not?" questions. In the case 
of the first three rules of INCREASE, the "how?" question Is 
perfectly well-formed. The program need only look at what 
is to be INCREASED without worrying about reasons Ml*. It 
shouldn't be done. There axe. no reasons, because the rules 
are valid for any case the program can encounter. Thus, 
the program can always go ahead and try the INCREASE. It 



Page 90 

can either fall (1) (as In the case of iNCREASlng a 
constant, for example) or It can set up the next subgoal 
(usually another INCREASE *G0AL) — all without worrying about 
"should" and "shouldn't". 

On the other hand, rules (U) and (5) 
must worry about "should" and "shouldn't" before setting up 
the next subgoal. Perhaps the user does not Intend for the 
INCREASE to take place. Thus, INCREASE must ask the "why 
not?" question before It proceeds. 



k.k.2 Jhs. voice of REASON 

We saw In the previous section that the 
use of INCREASE to ask the "how?" question leads directly 
to the need for the "why not?" question. As usual, the 
program frames this question as a *G0AL. That Is, given the 
♦GOAL of INCREASlng an *ACTIVITY "A" by "m" occurences In 
TIME-SLICE "n": 

(*G0AL (INCREASE A m n)) 



(1) A failure of this kind Is automatically for a "GOOD 
REASON" — see sections 2 and ii.it. 2.1. 



Page 91 
the program Immediately forms the *GOAL 

(*GOAL (SCHEDULE m A n)) 
to ascertain whether or not INCREASE should proceed. 

SCHEDULE'S job Is to examine 
SIMULATION-HISTORY and the user's model to determine why the 
change suggested by INCREASE was not originally part of 
SIMULATION-HISTORY. After all, since It presumably leads to 
the desired state, why didn't the user cause the state 
suggested by INCREASE In the first place? 

There are two kinds of reasons for the 
user's not causing the suggested state to occur Initially. 
A GOOD REASON Is that he deliberately Intends (for reasons 
best known to himself) the model not to allow that state. 
A BAD REASON is that the Interaction of the submodels has 
caused a constraint which disallows the state. A BAD REASON 
Is not a bug. It simply Implies that a constraint is due to 
submodel Interaction and not user Intention. However, given 
the bug philosophy of section 3, the program treats a BAD 
REASON as "suspicious" — a cause for further Investigation. 

In this section we examine the way the 
program distinguishes GOOD REASON'S from BAD REASON'S (and 
the way It classifies BAD REASON'S). The next subsection 
discusses the program's model of user intent-- 1 .e . , Its 
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method for discerning GOOD REASON'S. After this, we 
classify BAD REASON'S along the lines of the three 
"Interaction bugs" presented In section 3, 

«i.U.2.1 SQQB. E&ASMis. 

At each stage of the debugging process, 
the program Is trying to change an envl ronment. . .by using a 
resource, Inserting a new submodel ,etc. In order to do 
this, the program must face the question of whether or not 
the change shoul d (In terms of user Intentions) be made. 
Of course, It Is unreasonable to expect the user to have to 
tell the program at each step what should and should not be 
changed. In fact, given the philosophy of section 3, It Is 
very unlikely that the user could provide this Information 
if he wanted to. Thus, the program needs some sort of 
theory of which of the constraints found In 
SIMULATION-HISTORY are user-Intended and which are there 
because of a possible bug In the model. 

Going back to sections 1.3.1 and 3, we 
recall the previous assumptions about user Intentions: the 
user has a good understanding of each submodel, but only a 
very weak understanding of how submodels Interact to achieve 
an overall goal. Thus, the program can assume, at least 
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temporarily, that all Information fn the simulation history 
which Is derived directly from user statements about an 
Individual submodel Is user-Intended. All other Information 
Is necessarily the result of submodel Interaction and Is 
therefore suspect. The programming task Is to Interpret 
this simple theory (1) of user Intention In terms of the 
deductive mechanisms and SIMULATION-HISTORY. 

Everything In an MSL specification 
pertains only to a specific submodel; this, In fact, was a 
design criterion (see k.l). Thus, everything so far Is 
user-Intended, by our principle of locality. But this Is 
only static Information. Once the model Is simulated, some 
of this static local Information gives rise to Interaction 
between submodels. The question then becomes one of 
determing how locality Is preserved in the dynamic behavior 
of the model. That is, what's local about 
SIMULATION-HISTORY? 

According to I+.3, the answer seems to be 
that the TIME-SLICE Is used by the program as a "local 



(1) This theory Is of course quite liberal In Its suggestion 
of "suspect" constraints. At this stage, this seems to be 
the best strategy. The deductive mechanisms are capable of 
eliminating non-bugs rather easily so that things don't blow 
up (see section 2). However, If really large models were 
used, a better theory would be necessary to avoid smothering 
the program with possible leads (see section k.5). 
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envl ronment". . .but why? The TIME-SLICE preserves locality 
because direct user policy Is at the TIME-SLICE level. 
Scheduling decisions set certain *ACTIVITY's to occur In 
certain TIME-SLICE's (see description of *SCHEDULE In fc.l). 
♦PREREQUISITES are checked at the TIME-SLICE level, *OUTPUT 
occurs at the TIME-SLICE level, *FUNCTION's are cal led, 
♦EVENT'S triggered, etc. —all at the TIME-SLICE level. All 
of the direct user decisions, as specified by the static 
Information In the MSL, affect the simulation at the 
TIME-SLICE level. Therefore, the program takes a constraint 
to be local (and thus user-Intended) If It depends only on 
what happens In a single TIME-SLICE. 

Now I mentioned In 1.3.1 that the models 
used In this thesis are especially Interactive. 
Furthermore, as I said above, the criteria for suggesting 
unintended constraints can afford to be llberal--we would 
rather suggest wrong bugs than miss a possible bug. Thus, 
we would expect there to be few local "user-Intended" 
constraints and many non-local "suspect" constraints. This 
Is Indeed the case. The resources present In any TIME-SLICE 
are dependent on the action of the model over many 
TIME-SLICE's and are thus non-local. Similarly, the timing 
of *ACTIVITY's which do not contain *SCHEDULE specifications 
becomes resource-dependent and thus non-local. *EVENT 
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occurences are specified by probabilistic functions of 
resources and are thus non-local. Finally/ higher-order 
constraints like coincident presence of several resources 
span several TIME-SUCE's (see k.3) and are, almost by 
definition, non-local. These non-local constraints give 
rise to the BAD-REASON's discussed In the next two 
subsections. For now, let's mention the few GOOD REASON'S 
that exist. 

Most GOOD REASON'S concern constraints 
that arise from *SCHEDULE constructs. If the change 
requested by INCREASE would violate the *ACTIVITY's 
♦SCHEDULE for that TIME-SLICE, SCHEDULE denies the request 
for GOOD-REASON (1) . Thus, If, as In section 2, there are 
three ADVERTISING *ACTIVITY's already In a TIME-SLICE and 
ADVERTISING contains the modifier 

(•SCHEDULE 3) 

SCHEDULE will deny any request to up the amount of 
ADVERTISING In that TIME-SLICE. Similarly, SCHEDULE views 
the other avatars of *SCHEDULE (see k.l) as 
GOOD-RE ASON-genera tors. 

The other kinds of GOOD REASON'S are 



(1) There Is one exception to this which will be discussed 
In the next subsection. 
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those that are based on "fact" or are "true by definition" 
(see the first three rules of INCREASE In k.k.l). Thus, 
SCHEDULE will deny attempts to schedule In negative time, 
Increase constants, etc. for GOOD REASON. Actually, these 
REASON'S can be viewed as being based on the "common sense 
knowledge" the user has In addition to his knowledge about 
submodels. That Is, the user directly Intends his model to 
be "sensible" as well as to be In accordance with known 
submodel constraints. 

Thus, GOOD REASON'S apply to constraints 
which depend only on single TIME-SLICE Information, I.e., 
which reflect the local Ity which Is characteristic of user 
Intention. We now go on to Investigate the way In which the 
program deals with non-local constraints. 

u.u.2.2 lasic. MP. B£A£OiLLs. 

If the program cannot find a GOOD REASON 
for a constraint, It must attribute Its existence to a BAD 
REASON. From the "Interaction bug" philosophy of section 3 
we see that the user's understanding of his model falters In 
the three critical areas mentioned at the beginning of this 
section: 
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(1) the effects of resource competition among submodels 

(2) timing effects of submodels 

(3) the effects of higher-order constraints 

If a constraint Is there for no GOOD REASON, the program 
considers the possibility that the constraint arose 
unintentionally from one of these three misunderstandings. 
It will therefore try to come up with a BAD REASON for the 
constraint's existence so that It can Inform the debugger of 
the possible anomaly (see section k.k.3). This section 

will consider the BAD REASON'S related to the first two 
kinds of Interaction. These BAD REASON'S form the basis for 
BAD REASON'S arising from higher-order Interdependencles--as 
discussed In fc.4.2.3. Now, to continue with our favorite 
process, the SCHEDULE *GOAL was just seeing why the desired 
♦ACTIVITY wasn't scheduled In that TIME-SLICE In the first 
place... 

Since the user didn't specifically ask 
for the *ACTIVITY not to be scheduled, there can be only two 
reasons why the *ACTIVITY wasn't there: 

(1) some of Its prerequisite resources weren't present 
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(2) It Is dependent on an *EVENT that didn't occur 

Thus, the program first checks out the resource situation In 
the TIME-SLICE. If the resources are not sufficient to 
support the *ACTIVITY, there can be two reasons why: 

(1) the resources were available In the TIME-SLICE but 
were used-up by higher-priority ♦ACTIVITY 1 s before the 
•ACTIVITY In question got a chance at them 

(2) the resources just ali^t there 

To check out the first possibility, the program looks at the 
status of the higher-priority *ACTIVITY , s In the TIME-SLICE. 
If any of these *ACTlVITY*s Indeed "stole" resources which 
would have allowed scheduling of the desired *ACTIVITY, 
their names are collected and the BAD REASON 

(PRIORITY-RESOURCE-BOUND (<names of offending *ACTIVITY's>) ) 

Is recorded. 

If no higher-priority *ACTlVITY»s stole 
the resources, then the resources must just have been absent 
from the TIME-SLICE In the f I rst place. The ubiquitous 
two possible reasons: 
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(1) The *ACTIVITY's which *OUTPUT the desired resources 
weren't scheduled until It was too late for the 
resources to be available In the TIME-SLICE 

(2) The *ACTIVITY's which *OUTPUT the desired resources 
were scheduled too early and the resources were 
gobbled up by higher-priority *ACTIVITY's In the 
Intervening TIME-SUCE's 

Of course, In either Instance, the user may have Intended 
this to be the case (well we know how to check that out...). 
On the other hand, the *0UTPUT *ACTIVITY's may have ended up 
In the wrong place because of the user's poor understanding 
of timing effects (1) —a BAD REASON. To determine which Is 
the case, the program proceeds as follows. It first finds 
out what *ACTIVITY's *OUTPUT the desired resources and 
checks to see If they were scheduled too late to do the 
desired *ACTIVITY any good. Then, It sees whether the 
♦OUTPUT *ACTIVITY*s were "late" for GOOD REASON. If not, It 
notes a BAD REASON: 



(1) Note that the "Interaction Information" about timing Is 
Implicit In the resources. That Is, there are no explicit 
timer-alarms to say when something Is too late or too early. 
The only evidence of a timing error In the model will be 
found In the levels of particular resources over time. 
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(RESOURCE-BOUND (TOO LATE (<names 
of offending *ACTIVITY's>)>) 



If there are no "late" ♦ACTIVITY'S, or If the *ACTIVITY's 
were late for GOOD REASON , the program looks back up the 
SIMULATION-HISTORY for two things: *ACTIVITY's which *0UTPUT 
the needed resources scheduled "too early" for no GOOD 
REASON and "Interloping" •ACTIVITY'S of higher priority 
which stole the needed resources. If both of these things 
exist, the program notes: 

(RESOURCE-BOUND (TOO-EARLY (<names of offending *ACTIVITY's) 
(<names of Interloping •ACTIVITY' s>) )) 

Thus, the PRIORITY-RESOURCE-BOUND and 
RESOURCE-BOUND BAD REASON'S take care of the case In which 
the * ACTIVITY cannot be scheduled because of a lack of 
prerequisite resources (1) . This leaves the other case In 
which the *ACTIVITY could not be scheduled because It Is 



(1) As discussed previously, the program would try to 
alleviate this defflclency with an appropriate INCREASE 
*G0AL. The reason for this Is to make sure that the program 
traces through the entire Interaction path: after all, this 
resource defflclency could just be the result of an earlier 
decision which reflects the actual bug. More on this In 
h.k.3. 
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dependent on an *EVENT that didn't occur. 

The program can easily recognize this 
second case because It can only arise from the 

(♦SCHEDULE (ON <*EVENT-name>) ) 

specification (see 4.1) . If the specified *EVENT did not 
occur In the TIME-SLICE, the desired ♦ACTIVITY could not be 
scheduled. Now, If the program were acting like it did 
before, It would try to find out "why" the *EVENT didn't 
take place In the TIME-SLICE. However, this Is 
inappropriate for *EVENT's, which, after all, model 
occurences which are beyond the modeller's direct control. 
Of course, this raises the question of why a modeller would 
make an *ACTIVITY dependent on an *EVENT In the first place. 
Indeed, the program becomes suspicious: It Is possible that 
because of the user's poor understanding of timing effects, 
the *EVENT dependency (plus the time needed by the 
♦ACTIVITY) will cause the ♦ACTIVITY to take effect at the 
wrong time — usually too late (1) . The program checks out 



(1) The most common cause of this ♦EVENT-dependency Is the 
"f I re-f Ightlng" approach to solving problems: when the event 
occurs, start doing something about It. (This, Is, In fact, 
the problem In the example of section 2: HIRING Is dependent 
on QUITTING.) Note that this BAD REASON Is the exception 
to the "If ♦SCHEDULE says It's okay, It's okay" dictum 
referred to earlier. 
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this possibility by looking up and down SIMULATION-HISTORY 
to see If the * ACTIVITY was scheduled "too late" or "too 
early". If either of these Is the case, the program notes a 
BAD REASON* 

(♦EVENT-TRIGGEREO-SCHEDULE <offendlng *ACTIVITY> 
<"TO0 LATE" or "TOO £A*LY">) 

If neither of these Is the case, the program simply 
terminates Its 1 Ine of attack (1) on 

(*EVENT-TRIGGERED-SCHEDUIE) 

and goes away mumbl Ing to Itself (actually, this would be 

the first "GOOD REASON" It looks at after all the BAD 
REASON'S were checked by the debugger}. 

Well, this wraps up the "basic BAD 
REASON's" arising from poor understand Ing of resource 

conflict and timing effects. Now we go on to see how 

misunderstanding of higher-order constraints leads to the 
use of these same BAD REASON ' s In an expanded context. 



(1) Note that unlike the other BAD REASON'S, this one causes 
the line of attack to terminate — no further Investigation Is 
possible (see k.k.3). 
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Up until now (except for part of 4.3), I 
have over-slmpl If led the Interactive behavior of submodels 
for the purposes of discussion. Specifically, I have 
pretended that a submodel can depend on only one other 
submodel for its sources of input. Thus, my *ACTIVITY's 
have had only one unfilled ♦PREREQUISITE, my *FUNCTION's 
only one *ARGUMENT. This Is of course quite unrealistic, 
and not a real restriction of MSL. In this section I remove 
this artificial restriction. 

The Introduction of multiple dependency 
brings up the issue of higher-order constraints. As we saw 
in 4.3, when submodels depend on several other submodels for 
input, the problem-solver must take into account the 
interrelationship of the Input *ACTIVITY's. The input 
♦ACTIVITY'S are In fact operating under a "higher-order 
constraint" (see section 3.2) — they must combine to provide 
resources for a single *ACTIVITY (or *FUMCTI0N) at a certain 
time . This higher-order constraint Is modelled by forcing 
the Input *ACTIVITY's to share a local constraint 
environment (see 4.3). That is, all *ACTIVITY's sharing a 
higher-order constraint must be scheduled not only in 
accordance with their own needs, but also with the needs of 
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the *ACTIVITY or *FUNCTI0N that depends on them. There are 
two types of environment-sharing, reflected by two types of 
♦GOAL's to handle the higher-order dependencies. The first 
of these Is ♦AND, the expression of the way ♦ACTIVITY'S 
depend on each other when their higher-order constraint Is 
another *ACTIV!TY. The second Is ♦GROUP, which models the 
*ACTIVITY-*FUNCTION dependency. 

♦AND dependency arises from ♦ACTIVITY'S 
that look like 

(♦ACTIVITY SALES-CALL 

(♦PREREQUISITES 

(♦AND 

(♦PRESENT (1000 CASH)) 

(♦PRESENT (1 UNIT)) 

(♦PRESENT (SOME 

SALESMAN)))) 

* 

That Is, SALES-CALL depends on the submodels which ♦OUTPUT 
CASH, UNIT, and SALESMAN. AIL of these ♦OUTPUT'S must be 
present at once (I.e., In the same TIME-SLICE). Thus, any 
♦GOAL which tries to schedule a new SALES-CALL ♦ACTIVITY 
must take this Into account. Specifically, If the resources 
are not available, a_L± of the ♦OUTPUT ♦ACTIVITY'S Involved 
must be scheduled. That Is, given the ♦GOAL 

(♦GOAL (INCREASE SALES-CALL m n)) 
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and assuming none of the necessary resources are on hand (1) 
, the program must generate the subgoal 

(♦GOAL 

(•AND 

(•GOAL (INCREASE CASH j n)) 
(*G0AL (INCREASE UNIT k n)) 
(•GOAL (INCREASE SALESMAN 1 n)) 

)) 

Now, just as before, the program must be 
careful not to INCREASE things contrary to the Intentions of 
the user. Again, It uses the SCHEDULE *G0AL to find out the 
REASON for constraints. However, the SCHEDULE *GOAL cannot 
simply check out each INCREASE *GOAL Independently as 
before. The INCREASE *G0AL's are now Interdependent and 
must be treated as such. So now, finding GOOD and BAD 
REASON'S Is a whole new game. 

Not really. Fortunately, the process 
Isn't very different, especially In the case of *AND. First 
of all, examination of the whole GOOD REASON-f Indlng 
philosophy and Implementation will show that It Is 
completely unaffected by higher-order Interdependences. 
This Is almost by definition: GOOD REASON'S pertain to 
Individual submodels and TIME-SLICE's, while higher-order 



(1) In section 2 I kept higher-order constraints out of the 
picture by buffering away dependencies. Thus, In the case 
of SALES-CALL, all resources except SALESMAN were available 
already (see section 2). 
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interdependencles transcend these boundaries of local I ty. 
Thus, SCHEDULE'S GOOD REASONIng processes are still the 
same. Certainly, however, the BAD REASONIng Is different. 
But most of the differences have been taken care of already 
by the environment-sharing discussed In «t.3. That Is, the 
effects of higher-order constraints on resource conflicts 
and time dependencies are already reflected In the way *AND 
♦GOAL'S are set up and processed—the higher-order 
Interdependency Is already modelled. For example. If 
satisfying one component *G0AL steals resources from another 
or disturbs the timing of another, the shared environment 
will make this Interaction explicit: the resources needed by 
each *G0AL are recorded separately so that the effects of 
everything done In the *AND environment can be traced to the 
proper source. 

All this Is saying that all SCHEDULE has 

to do about *AND's Is to realize that It Is In a shared 

environment and attribute BAD REASON'S to the effects of 

sharing. Thus, the searches for higher-priority *ACTIVITY's 

and timing problems which were previously carried out only 

In a single TIME-SLICE are now carried out In the whole *AND 

environment. The "new" BAD REASON'S they generate look like 

(PRIORITY-RESOURCE-BOUND (<names of offending 
*ACTIVITY's>) *AND-M0DE) 

(RESOURCE-BOUND (TOO-EARLY (<names 
of offending *ACTI VITY's>) 
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♦AND-MODE (<names of Interloping *ACT!VITY's 

In the *AND envl ronment> ) (<names of other 

Interloping *ACTI VITY's>) ) 

etc. 

The theme here Is that most of the work 
for finding higher-order BAD-REASON's In the *AND case was 
done by setting up the *AND environment In the first place. 
That Is / the Interdependency Is already explicitly modelled 
by the way *AND *G0AL's work, and need only be checked 
through by SCHEDULE to find the appropriate BAD REASON'S. 
This theme Is elaborated for the *GR0UP case. 

In ii.tt.l I postponed the Issue of 
INCREASIng *FUNCTI0N's by attributing this task to a 
separate INCREASE-FUNCTION *G0AL-type. The job of 
INCREASE-FUNCTION Is to figure out a way to Increase the 
value *RETURN'ed by a *FUNCTI0N by changing the values of 
its *ARGUMENTS (thus. It Is completely analogous to 
INCREASE). Obviously, this problem is extremely difficult 
for a large class of functions. Fortunately, the functions 
needed In business games, and, indeed, In most of business 
processing, are of a very simple nature (1) . MSL currently 



(1) The mathematics of management science --! .e. . mathematics 
meant to model systems and decisions — can be quite 
sophisticated, but this ts not business processing. Indeed, 
even in a business game, the probability-handling can get 
tricky. But all of this Is built into MSL— the user can 
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allows the representation of only two kinds of functional 
dependencies: tables and linear functions of a few 
variables. The mathematical techniques for Increasing these 
♦FUNCTION'S are simple and are not of Interest here. The 
Interesting part of ♦FUNCTION'S for this discussion Is they 
are responsible for the second kind. of higher-order 
Interdependency. 

We just saw how the relation between 
♦PREREQUISITES and *0UTPUT's causes *AND Interdependency. 
Similarly, the relation between ♦ARGUMENTS and ♦RETURN 'ed 
value causes *GR0UP Interdepency. In the *AND case, the 
Interdependency was that aJJ. *PREREQUISITES must be present 
In the proper quantities In a single TIME-SLICE for the 
♦ACTIVITY to be Initiated. ♦GROUP Interdependency Is 
weaker. We know only that some comb I nation ^f changes to 
the components will bring about the desired change to the 
higher-order constraint. That Is, each subgoal can 
contribute an unspecified amount to the success of the 
overall ♦GOAL. Perhaps the Increase of only one of the 
♦ARGUMENT resources wl 1 1 suffice to Increase the ♦RETURN'ed 
value. Or, all may be necessary— making the ♦GROUP an ♦AND 
at the extreme. 

Now the program must model this kind of 



only define simple functions which use the probabl 1 Ity 
machinery. 
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fnterdependency when it tries to INCREASE *FUNCTION's. 
Furthermore / In trying to solve the INCREASE-FUNCTION 
problem, It must go about the task pretty much the same way 
organizations do In order to run Into the same kind of 
Interactive behavior. That Is, the Interaction Involved In 
a kind of breadth-first approach to the problem (Increase 
each *ARGUMENT resource a little In turn until the 
*RETURN'ed value has been IMCREASE'd the desired amount) 
causes very different subgoal Interaction than, say, a 
depth-first approach (Increase each *ARGUMENT as much as 
possible separately to see how much It helps to INCREASE the 
♦FUNCTION). The differences are In which subgoals are 
allowed to be achieved at the expense of others (1) , the 
range of subgoals tried, and the extent to which each 
subgoal Is exercised (2) . Clearly, different 
Interdependences are tapped by different subgoal attack 
methods. 

So the program must try to overcome the 



(1) Unlike *AND, this Is allowed because not all *GR0UP'ed 
subgoals must be achieved. The only requirement Is that all 
of the subgoals which eventually succeed must share the same 
local constraint environment (otherwise the construct 
doesn t model higher-order interdependency) . 

(2) Note that this need to model the organization's 
problem-solving method was not present in the *AND case. 
Since all subgoals must be achieved as stated, no 

resource-stealing" Is allowed among them and all of them 
must be fully tried and executed. 
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higher-order constraint of Increasing a 
functionally-determined value the same way organizations do. 
Obviously, this Is a tall order. Firs* of all, functional 
relationships are usually implicit In organizations, not 
explicit as In MSL— so It's hard to see what organizations 
do about them. Second, It Is reasonable to assume that 
different organizations attack different functional problems 
In different ways at different times. Finally, It Is 
possible that the actual process Is not pre-defined at all 
In many cases, but Is Instead made-up and modified during 
the course of each problem's solution. What I am trying to 
say by all of this Is that I'm not about to solve the whole 
problem or even a very big part of It... 

What I have done Is to program a single, 
slightly sophisticated method of attacking higher-order 
functional constraints which attempts to model one way In 
which an organization might do It. It should be seen as an 
experiment for demonstrating the approach of the program In 
dealing with this kind of constraint, not a fully developed 
piece of the system. This part of the program. Incorporated 
In INCREASE-FUNCTION, works as follows: given a *G0AL of the 
form 

(♦GOAL 

(♦GROUP , , ,,, 

(♦GOAL (INCREASE argumentl tlmel)) 

(♦GOAL (INCREASE argument2 tlme2)) 



Page 111 

• ■ 

)) 

the program takes the first *GOAL 

(*GOAL (INCREASE argumentl tlmel)) 

and tries to INCREASE argumentl the minimum possible amount 
as a "feasibility study". It carries the *GOAL all the way 
to completion, If It can. If the *GOAL Is unsuccessful 
(for GOOD REASON), It Is wlthdrwan from the *GROUP and the 
program does a "feasibility study" on the next *GOAL In the 
♦GROUP. If no "feasibility study" Is successful, the whole 
*GROUP naturally falls. Now, If any of the "studies" are 
successful, the program will keep attacking the studied line 
until It falls. When this happens, I.e., when the 
particular *ARGUMENT has been INCREASE'd as much as 
possible, the program considers Itself to have a "partial 
success". That Is, the effect of the INCREASE'd *ARGUMENT 
Is now calculated Into the overall *GROUP *GOAL, so that a 
new *GROUP *G0AL Is formed such that 

(1) The fully INCREASE'd *GOAL Is no longer In the 
•GROUP 

(2) The overall *GOAL Is reduced by the amount 
contributed by the successfully INCREASE'd *G0AL 
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In this Dfiw. *GROUP environment, the other *GOAL's are 
similarly processed until success (or failure) occurs. 

All of this hopefully goes toward 
modelling the way an organization attacks this kind of 
problem: by checking out and eliminating possibilities one 
by one/ and pushing winning lines as far as possible to 
achieve the overall *GOAL. As Intimated In fc.3, the process 
Is modelled (like *AND) by the proper sharing of 
environments. Obviously, the environment-hackery for 
•GROUP'S is a bit more complicated than for *AND (for 
example, It must Incorporate the notion of "partial success" 
and the fact that all the eventually successful *G0AL's and 
only the eventually successful *GOAL's share the same local 
constraint environment). The question for us here Is how 
this affects the GOOD and BAD REASON Jng process. 

Again, the answer Is "not all that 
much". As with the *AND case, the only difference Is that 
the BAD REASON'S differentiate between constraints caused by 
higher-order Interaction and those caused by other kinds of 
Interaction. This Is again just a matter of tracing through 
the explicit relationships set up In the *G0AL's environment 
structure. As far as actual BAD REASON'S for constraints 
go, *GR0UP only adds two (minor) new wrinkles. First of 
all. It will make a special notation If the constraint comes 
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up during a feasibility trial. Second, ft carefully notes 
which *GR0UP *GOAL's have already succeeded when the 
constraint comes up. These are just convenience factors 
which the bug-finder uses when suggesting *GROUP bugs to the 
user; It wants to make clear exactly what the program was 
doing when It ran Into the constraint. This Is Important, 
because, as mentioned above, different Interaction occurs 
depending on exactly what the program does. 

This brings up a final Important point. 
♦GROUP BAD REASON'S are perhaps the weakest In the REASON 
repertoire because they depend directly on the actual 
exploration methods used. That Is, the program might 
suggest a BAD REASON which the user may never really 
encounter because of the way his organization handles 
functional dependencies. Thus, the debugger saves 
*GROUP-type bugs for last. Nonetheless, I think that it is 
very Important to Include this kind of REASONlng In the 
debugger: *GROUP-style dependencies are pervasive In 
organizations. Furthermore, they point the way toward 
modelling more sophisticated kinds of submodel -submodel 
Interactions . The weakness of the *GROUP method In this 
program Is Its Incompleteness, not Its basic concept. 

This section has catalogued all of the 
BAD REASON'S generated by the program. Now we finally get 
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around to finishing the bug story by showing how the BAD 
REASON'S are used to suggest the actual model bugs. 



h.k.3 lbs. do st -mortem recrlmlnat lOHS 

So far, the debugger has been left with 
a bunch of GOOD and BAD REASON'S for constraints. It Is now 
time to turn these Into bug suggestions. So, let's see what 
the REASON'S mean to the debugger. If the problem-solver Is 
faced with a BAD REASON for a constraint, It knows that the 
constraint Is based on submodel Interaction. Its job Is to 
explore that Interaction. Therefore, when SCHEDULE returns 
a BAD REASON, the problem-solver considers It a cause for 
further Investigation. In this way, It carries the 
perturbation as far as It can— tracing the Interaction 
patterns to their roots. 

GOOD REASON'S are the "roots" that stop 
this search through the interaction path. They Imply that 
the constraint blocking the path Is not due to Interaction, 
but rather to direct user Intent. The program should not 
disturb user Intent, since its only purpose In changing the 
environment Is to debug the existing model. It now has a 
GOOD REASON to stop changing the environment, so It stops. 
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Its current line of attack is said to "fall" (In Its attempt 
to bring about the desired change). Thus / the 
problem-solver's activities leave a line of *G0AL's attached 
to BAD REASON'S ending In a *G0AL attached to a GOOD REASON 
(1) . Now what does all of this have to do with debugging? 
Simply this: the program has now tried to overcome every 
Interaction-based constraint In the way of producing the 
user's desired state. It has reached a user-desired 
constraint which Is the root cause of all of the 
interaction-based constraints. Therefore, It has reached 
the end of the line and cannot produce Xhs. user's desj red 
state . There can be three reasons for this state of 
affairs: 

(1) The user's desired state is off-base: he has set 
the model an Impossible task 

(2) One of the user's original Intentions Is wrong; 
I.e./ one of the root constraints Is the bug 

(3) One or more of the interaction-based constraints 
between the root constraints and the desired state are 
incorrect: the model has an Interaction bug 

It is obvious from what has been said before that the 
program thinks that possibility (3) Is the most likely. It 
therefore suggests that one or more of the interactive 
constraints (noted by BAD REASON'S) are caused by the bug. 



(1) Except for the *EVENT-TRIGGERED-SCHEDULE case discussed 
in U.I*. 2. 3. 
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That Is, given that the Interaction constraints are wrongly 
causing the discrepancy, the debugger's job Is to find the 
part of the model which gives rise to the faulty 
constraints. This Is then suggested as the "bug" In the 
user's model. If the user doesn't agree with any of the 
program's suggestions based on possibility (3), the program 
falls back on (2), and finally (1). Anyway, let's pick up 
the process again at the posslbllty (3) suggestion phase. 

The program now has the location of the 
bug bracketed between the beginning and end of a "line of 
attack". Furthermore, the submodels which could have caused 
the bug have been narrowed down to a relatively smalll 
"Interaction group" (the union of all submodels mentioned In 
the bracket) (1) . The program must now pick out the 



(1) The size of the "bracket" and "Interaction group" of 
course depends on the model. However, In the experience I 
have had, the relevant groups have been small: a few BAD 
REASON'S and thus slightly more possible submodels. In the 
case of higher-order stuff, the group gets somewhat larger. 
There Is no reason to expect brackets or Interaction groups 
to get much larger for larger models: the key factor In 
determining their size Is the amount of control the user 
exercises over his model (In MSL, the extent to which things 
are determined by *SCHEDULE* s) . Control means GOOD REASON'S 
and thus short paths between Initial manifestations of a 
discrepancy and GOOD REASON'S to close the bracket. Control 
also means smaller groups of submodels which can affect the 
timing and resource-allocation of other submodels. Since 
managers (and modellers) exert considerable control over 
their systems, the amount of uncontrolled Interaction 
possible In any realistic model Is probably quite 
reasonable-sized. This In turn means that brackets and 
Interaction groups should also stay reasonable-sized. 



a 
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submodels In the "group" which caused the BAD constraints In 

the "bracket". 

Sometimes this is quite easy: all of the 

BAD REASON'S are traceable to a single submodel Interaction. 

Examples of this are the *EVENT which triggers an *ACTIV|TY 

at the wrong time, the *ACTIVITY which constantly steals 

resources from other necessary *ACTIVITY's, and the 

♦ACTIVITY which Is always too late (too early) to allow 

nother *ACTIVITY to be Initiated on time. The program 

looks for these single-cause Interactions by scanning the 

BAD REASON'S In the bracket, looking for "give-away" BAD 

REASON'S like *EVENT-DEPENDENT-SCHEDULE or consistencies in 

the "offending *ACTI VITY 1 s" and "interloping *ACTIVITY's" 

listings. If, In the process of examining the bracket, the 

debugger finds a single such cause for the BAD REASON'S of 

the bracket, It immediately labels the faulty interaction 

(I.e., the submodels Involved in the interaction) as lhe_ bug 

for that bracket, and files it away. Often, however, In 

looking at the BAD REASON'S of a bracket, the program finds 

that a particular BAD REASON could have been caused by any 

of several interactions. For example, *ACTIVITY A couldn't 

be scheduled because B stole Its resources, or because C 

caused D to be late so that D couldn't provide the necessary 

resources for A. The program handles this by noting each 
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cause separately as a bug. 

Sometimes this straightforward process 
breaks down: the program Is unable to pick out the cause for 
the BAD constraints of a bracket (this happens mostly In 
♦AND's and (especially) *GROUP's). Currently, the program 
simply presents the troublesome bracket to the user telling 
him that "there's something wrong In there". I consider 
this an Incomplete part of the program (see. U.5). 

When the program has found the bug (or 
the few bugs) for each bracket, It presents them to the user 
In order of "likelihood". The debugger's model of the 
likelihood that a suggested bug Is actually a bug In the 
model Is 

(1) The more specific the suggested bug, the more 
likely It Is that It Is genuine; thus, bugs like 
*EVENT-DEPENDENT-SCHEDUtE which correspond to a single 
BAD Interaction are suggested first. 

(2) The more definite a suggested bug, the more likely 
It Is; I.e., brackets which contain a single possible 
bug are suggested before those with multiple bugs, 
which are In turn before those which are just brackets 
with the "something's wrong" tag. 
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(3) The more Interactions encompassed by a single bug, 
the more likely It Is; this Is just a recursive 
application of Murphy's law... the more Interaction 
decisions a user has to make, the more he'll blow— thus 
*AND bugs (1) and long timing chain bugs (A was late 
for B was late for C was...) come early. 

U) Timing bugs are more likely than resource-conflict 
bugs; PRIORITY determinations are much closer to local 
specifications, and are thus more likely to be 
user-intended than the mul t l-TIME-SLICE machinations of 
a timing bug. 

(5) *GROUP bugs are saved for last. 

(6) After all of the bugs due to Interaction are gone, 
the program works on the second posslbl 1 I ty stated 
above— I.e., It starts suggesting that the GOOD 
constraints are faulty (I.e., wrong *SCHEDULE 
specification, etc.); It starts with the 
*EVENT-DEPENDENT-*SCHEDULE fifiQD. REASON If It's 
around— It's suspicious. 



(1) *GROUP bugs would be here too, except, as I mentioned In 
father dubious C te meChan?Sm f ° r handl ,ng them fs 
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(7) The program suggests missing submodels (see k.S). 

Thus, the program goes through Its suggestion repertoire bug 
by bug, providing the user with an orderly statement of what 
the program thinks might be wrong with the model (see 
section 2 for the format of the suggestions). The user can 
always ask to see the Interaction path leading to a bug, the 
bracket of a bug, and any other bugs which pertain to a 
particular bracket. 

If the user does not agree with any of 
the bugs suggested, the program will suggest possibility 
(1): that his original *G0AL was wrong. If the user Is 
still unsatisfied after all this work, the program Informs 
him as to the location of his head and logs him out. 

d.5 ppn't confuse ID£ with XhS. facts 

Most of the program's knowledge about 
models Is contained In Its conceptions of MSL (Including, 
for example, Its Ideas of how to INCREASE MSL quantities) 
and Its notions of user Intent Ion--as discussed In k.k. 
However, as I mentioned In section 2, It Is useful from a 
debugging point of view to Include actual "world" knowledge 
of business games. Clearly, this knowledge can be used to 
suggest bugs which transcend the MSL specification. 
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This Is, In fact, the only use the 
current program has for WOBG knowledge. As shown In section 
2, the program has a facility for suggesting "missing" parts 
of an MSL specification. This comes from a (very simple) 
model of what an MSL model of a business game (1) could 
contain. The program simply checks at various points to 
see whether the addition of an *ACTIVITY could solve some 
problem (usually alleviate some defflclency) in the user's 
model. Thus, when there Is a lack of CASH In the sample run 
In section 2, the program notes that the addition of a 
FACTORING *ACTIVITY (see description In Appendix A and 
specification In Appendix B) could solve the problem. 

While this sort of thing Is certainly 
useful, It Is only a "zeroeth order" attempt at using world 
knowledge In debugging. A more Important use of WOBG 
knowledge would be to aid In finding bugs within the MSL 
specification (I.e., the same kind of bugs the program now 
finds). As I mentioned In k.k, a major determiner of the 
efficacy of the debugging program Is the number and size of 
the "brackets" which enclose possible bugs. In the current 
program, brackets are determined by the amount of 
uncontrolled interaction — I.e., a purely MSL-level 
criterion. In a more thorough-going approach, WOBG 



(1) In fact, It Is based entirely on the game in Appendix A. 
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knowledge could be used to determine which interactions are 
really natural and which are possible bugs (1) — thus 
limiting or even eliminating brackets. Also, WOBG knowledge 
could be used to suggest suspiciously specified ♦ACTIVITY'S, 
etc. 

The main reason that I have not 
exploited WOBG knowledge In these more sophisticated ways Is 
that It has not been necessary for the models I have 
investigated so far. Furthermore, It Is interesting to see 
how far a "doma In- Independent" (2) debugger can go toward 
finding bugs in MSL models. Thus, WOBG knowledge does not 
enter Into the main bug-finding process at all. Its sole 
use Is In suggesting the addition of *ACTIVITY's to the 
current model (3) . 



(1) This sort of thing Is actually found to some degree in 
the programs of Sussman |18| and Goldstein |5|. 

(2) See Sussman's discussion of the doma In- independence of 
HACKER | 18 |. 

(3) It operates off a WOBG database which will not be 
described here. It works a lot like MAPL 1101, and was In 
fact designed to be compatible with the larger MAPL database 
of Protosystem I (the WOB |9|). 
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5 Conclusion* 

I would like to use this concluding 
section to fit my model -debugging system Into the "big 
Picture", viewing It first as a debugging tool, and second 
as part of an automatic programming system. 

The approach of my debugging system 
should be seen as one method of the several which can be 
used by the human or machine problem-solver. The 
slmulate-and-lnvestlgate technique shown here Is useful for 
debugging poorly understood but easl 1 y model led systems. It 
requires the modeller's knowledge and lack of knowledge to 
be of a certain character, as outlined earlier. It Is also 
most useful for handling highly Interactive systems. If the 
problem domain Is very well understood, or If actions In It 
are basically Independent, other techniques are simpler and 
much better. 

Furhtermore, It should be stressed that 
the debugging methods of the program are quite naive In the 
context of a real (I.e., non-game) Interactive system. It 
Is almost certain that all of the techniques described here 
would have to be shored up with procedures based on 
knowledge of the problem domain (see k.5). Remember that 



Page 124 

the basic "smarts" of my system is In the exploration of the 
simulation history. In real life, this exploration phase Is 
usually preceded by some knowldgable guess work on the part 
of the debugger: almost all expert human debuggers 
(programmers, consul tants,etc. ) start their exploration for 
a bug with a good preconceived notion of the nature of the 
bug. This "notion" comes from the utilization of long 
experience about what kind of bugs are attached to what kind 
of problems; most debuggers know that only one or two things 
could possibly cause a bug at any given time In their 
exploration. No one yet knows how to encode this key 
experiential knowledge into a computer program. Certainly, 
no attempt has been made In this thesis. 

Thus, the program presented here, when 
viewed only as a general debugging technique, should be seen 
as part of a larger system: It fits In after an Initial 
"guesswork" phase (as one of several possibly applicable 
techniques) and just before a "weeding out" phase which 
makes thorough use of knowledge In the problem domain to 
narrow down the choice of possible bugs. 

The model -debugging needs of an 
automatic programming system are somewhat different. Here 
the user is Interested In expressing a model of his problem 
to the machine in such a way that he can be sure that the 
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machine understands It properly. Thus, after a phase of 
model specification aid at def Ine - tlme (1) , a 
model -debugging system like the one here can come In and 
demonstrate the APS's Idea of the model to the user's 
satisfaction (and help the user overcome any dlcrepancys) . 
The slmulate-and-lnvest Igate and domain- Independence 
philosophies of my system are wel 1 -adapted to this purpose: 
the system can afford to be an expert In Its own modelling 
language and do a great deal of exploration work In finding 
bugs. Furthermore, the user can tolerate a reasonable 
number of program-generated choices of bugs In his model If 
he can be certain of eventual understanding by the APS. 
Therefore, I think that the techniques used here might find 
direct application In automatic programming. 

Nonetheless, for a debugger to be truly 
useful, whether In an automatic programming or general 
artificial Intelligence environment, It must Incorporate the 
same kind of experiential debugging knowledge found In the 
human expert. This kind of stuff will surely be the basis 
of the next generation of debuggers which are now on the 
horizon. 



(1) See I 9| for Protosystem l's "activity expert modules". 
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Appendix A 



The following Is excerpted from the 
article "Business Games — Play One!" by G.R. Andl Inger In the 
Harvard Business Review for March-April, 1958 ( © The 
President and Fellows of Harvard University)— It Is 
reprinted by permission. 

It serves as an example of the kind of 
business games at which the program (and MSL) are directed. 
An MSL model of the game described here appears In Appendix 
B. 
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Business Games — Play One! 
Basic Objectives 

haclr ^h'^+t . Games are as old as man. Usually, their 

cttl h2 J CtlVe . ,S entertainment. The Business Management 
SJ5: S7~ er ' a?ms not at entertainment, but at learning 
exaZle^arer" 5 ° etWeen ft a " d a «~ Uke Monopoly, ffr 

--The degree to which ft approaches reality. 

t "~ The . de « re e to which the players* 

1nfull nCe '^ judgment ' and ski 11 -as opposed to luck- 
Influence the outcome. 

that- of i%!!] y t bu f! neSS game ls to serve a Purpose beyond 

fearnlnl fml tV™**"* ?? y ', there must be so ^ transfer of 
learning from the game situation to reality. While therp 

bus?ne b s V *Le S °tLt SUC f traMf 8 r fr ° m play '"« a ^neraHze" 
:." !!, L g am S. that mirrors "any company" and not a 

create? bln.ffJT a " execu J' v « could derive Infinitely 
Vuldtncr *Ef f, I f r? m a g ! me that Permits him to practice 
fninl? 8 he , dest I n y of h's own company or one In his own 
Industry-whlch unfortunately. Is unavailable at this al 
stage of business gaming. The success of specific war 
games, which the military has been using for years to 

hoTd" a g e rearorom. se S ^ UatI ? n ? 1 f0r tra,nfng ofV.ce^s/howeVr? 
dCe coSr"! Pr ° m ' Se f ° r slm " a r applications In business In 

in ,w>t„* •.. ^ T he Bus Iness Management Game Is a case 

w£r-™?n "* S J a r ted ?t ,n 1956 w,th the fd « a of applying 
?ea7we tUtll ™3m /° 5 usIness ' ,n the course of thf 
to develoo J'fTSJ 1 k'? ' an u retested the game many times 
to develop a fine balance between realism and playablllty 
The more cosely a game resembles real I ty, the more 
S™r S ?T ". becomes "«"tIl It Is no longer playable 
Hence, there Is a need to compromise. Also, we designed 

resulfTn suSd^n : e1at?vel * ■"*!.. No extreme stra?egycan 
success iJ udde " succe ss; yet players can gain outstanding 
not careful. g °° d enough -° r bankruptcy If they are 

oart-lv n , n u a L,i. «., The f ame ls P art] V deterministic and 
by the ac?lon ir^h S °T reSUltS are determined directly 
decrees ™£?.r? «- Payers; others are, to varying 

aegrees, subject to chance or probabf 1 I ty. The weleht of 
the elements of the game Is such that the longer the game! 
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the smalller the influence of luck. 

Rules of Play 



In this section I shall give a brief 
general description of each game element and the specific 
values, rules and probabilities that define each element In 
quantitative terms. Instructions for the umpires are 
Included at each point; but remember that they should not be 
given to the players. 

The Market 

The market Is made up of 2k customers. Each 
customer's potential is different; In any one time period, a 
few customers are not buying any units, while others may buy 
four or five units (at $10,000 per unit) JL£ a salesman is 
able to make a sale. 

The market is dynamic, so the customer 
potentials change. If the market is growing, they change 
upward; should the market be hit by a recession, however, 
they may drop drastically. The long-term trend of the 
market Is announced to the players; short term fluctuations 
are not. If a company Is Interested In finding out what the 
total market potential is In any time period, a $2000 
expenditure for market research will buy this Information 
from the umpires. 

The 2«* customers divide geographically 
Into four regions on the game board, each region containing 
six accounts. This geographical division allows the company 
to do local advertising (see the section on "Advertising the 
Product") and conduct market research In only one region at 
a time. Such market research, which tells a company the 
potent lal of each customer In the region and permits the 
pinpointing of the direct selling effort (see the section on 
"Marketing the Product"), may be obtained by paying the 
umpires $30,000 for "staff work." 

In addition to the separation Into 
geographical regions, the market breaks down Into one rural 
and two urban markets. The significance of this distinction 
Is that In an urban market, where a salesman can make more 
calls per day, he has two chances of making a sale during 
each time period, while In the rural market he has only one 
chance. 

If at the end of a year a company 
desires to find out what portion of the total market It has 
been able to capture, It may but a share-of-market 
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Information from the umpires for $2000. 

The umpire should: 

(1) Keep a list of all current account potentials. 



(2) Distribute a total customer 
comes to $360,000 at the beginning of the 
the 2k customers as follows: 

1 account 
3 accounts 
5 accounts 
13 accounts 

2 accounts 



potential , which 
game, at random to 



$40,000 

30,000 

20,000 

10,000 





(3) Depending on the economic climate determined 
In advance, change these starting potentials as the game 
progresses as follows: 



--For slow growth, chane one account each quarter 
at random. Move ahead on the random number table 
untfl a number between 01 and 2k appears, then add 
$10,000 to the potential of that account number. 

— For faster market growth, change two or three 
accounts In the same mannner as above for each 
quarter. 

— For a depression, change half or all of the 
accounts to zero for one or more quarters. 

(k) If a company decides to buy market Information 
(total potential, market research, or share of market), 
write the Information on a slip of paper and pass It to the 
company. 



Marketing the Product 



Units are 
In the 
two calls 



the 2k accounts 
salesman may make 
market, only one. 

In the 
manager of a company points 



sold by salesmen, who call on 
market. In an urban market a 
per quarter; and In a rural 



presence of an umpire, the sales 
to the accounts he wants to call 
on. The umpire will tell him, after examining the random 
number table, whether a sale Is made or not. How many 
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units are sold to a customer will depend on competitive 
action. The completed decision form, returned to the 
company at the end of the particular period, contains the 
actual sales results by accounts. 

Whenever a salesman has two calls, he 
must make the second call on a any of the three to eight 
accounts adjacent to the first square called on; that Is, he 
may not jump accross territories. If no sale Is made on the 
first call, he may, of course, call on the same account 
again during the same quarter. Furthermore, there Is no 
limit to the number of salesmen who may call on the same 
account In one time period. Between quarters, salesmen may 
be moved to any accounts that the company wishes to cover 
during the next quarter. 

Each time a salesman makes a call, he 
has a certain fixed probability of making a sale. This 
chance of making a sale may be Increased In one of three 
ways or a combination thereof: 

— A company may Intensify Its direct selling 
effort by having more than one salesman cover one 
account as described above. In such a case, If 
the first salesman makes a sale, the second one 
may move to any adjoining account for his calls. 

— A company may support the salesman's effort by 
advertising (see "Advertising the Product"). 

— A company may attempt to Improve Its product by 
spending more money for a research and development 
effort (see "Research and Development"). 

Every salesman costs $10,000 to hire and 
then $1000 per quarter In slary. (Since the product he will 
be selling Is a high-price, complicated unit, It takes one 
year to train a salesman before he can be sent out Into the 
field.) There Is a possibility that a salesman will 
resign. In which case the umpire Informs the company of this 
loss. 

The umpire should have the following 
Instructions for marketing: 

(1) Each period there Is a 5% chance of loss for 
each salesman. Move ahead on the random number table as 
many numbers as the company has salesmen; If one or more of 
these numbers Is .05 or less, the company loses one or more 
salesmen. 

(2) In an urban market, allow two calls per 
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quarter; In a rural market/ only one call. 



(3) A salesman always has a 25% chance of making a 
sale. For each call, examine the next number on the random 
number table. If the number fs 25 or less, then a sale has 
been made; If It Is 26 or more, no sale Is made. 

Advertising the Product 

Product advertising In any quarter 
Increases the salesmen's chances of amklng a sale. It 
covers only the region or regions (1,11,111, and IV on the 
game board) that the company designates, and Is effective In 
the current quarter only. Advertising costs $3000 per 
page, and a company may buy up to five pages of advertising 
In any region In any quarter. 

Here are the umpire's Instructions: 

For each sales call within the reglon(s) In which 
the company has advertised, go to the next number In the 
random number table and determine whether or not there Is a 
sale according to the probabilities In the following table. 
If the number Is the same or below the probability 
percentage, a sale Is made. 

Pages Amount Probability of a sale 

25% 
29 
35 
h2 
U8 
52 



Research and Development 



If a company can develop a superior 
product. It gains a competitive advantage. Usually, 
research and development have to be fairly continuous to 
achieve a product Improvement, but a "crash program" may 
yield results In a relatively short time. The minimum 
research effort per quarter costs $10,000, but a company may 
Invest more than that In multiples of $10,000. 

The umpire notifies the company 
Immediately when Its research and development program has 
produced results, and all units scheduled for production in 
that quarter are considered to be equipped with the 
Improvement. To find out the extent to which customers wll 









1 


$3,000 


2 


6,000 


3 


9,000 


1+ 


12,000 


5 


15,000 
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prefer an Improved product, $5,000 of market research 
(obtained from the umpires) Is needed. 

Of course, these ground rules can be 
altered to fit a company's situation more closely — just as 
the ground rules for other aspects of the Business 
Management Game can. A company manufacturing equipment for 
railroads may well want to use different units of research 
expenditure than would a company making dies for plastic 
products. The length of time necessary to get results from 
research also varies greatly from company to company, as 
does the cost of research to measure customer reactions to 
new products. These and other rules can—and In many cases 
should— be' tailored to the realities of the Industry. 

The umpires will tell a company as soon 
as a competing team Introduces an Improved product In the 
market. The players can then counter with a stepped-up 
marketing effort or a crash research and development 
p rog ram . 

If a company Is Interested In finding 
out the total Industry research and development expenditures 
for the past year, such Information Is available from the 
umpires for $1,000. 

In addition, the umpires should: 

(1) Maintain a cumulative account of each 
company's expenses. After each break In continuity (a 
quarter without any R & D expenditures) and after each 
product Improvement, start the accumulation over again. 

(2) Make appropriate revisions of the probability 
of Improvement. The cumulative dollar amount spent on 
research and development determines the chances a company 
has for obtaining a product Innovation. Examine the random 
number table; If the next number Is the same as or below the 
probability percentage, an Improvement Is achieved. 

Cumulative amount Probability of Improvement 

$10,000 0% 

20,000 

30,000 

i»0,000 2 

50,000 k 

60,000 7 

70,000 11 

80,000 15 

90,000 18 

100,000 and over 20 
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(3) Whenever a company achieves an Improved 
product, Increase all Its sales probability percentages by 
10. For example, If Company A has an Improved product, this 
Is the result: 



ProbabI 1 tty of sale 
Old product 25! 

Improved product +10 

35? 



If Company A 
region and has — * 
that region: 



my A spends $6000 on advertising In one 
an Improved product, this Is the result In 

Probabll Ity of sale 
Old product with 

two pages of advertising 35% 
Improved product +10 

CO As soon as all three companies have Improved 
products on the market, cancel the premium of 10 for all 
three. 

(5) If one company achieves two product 
Improvements before one or both of Its competitors have 
achieved any, Increase all Its sales probability percentages 
by 20. 

Increasing Production 

The Initial plant which each company 
must build costs $150,000, and hds a maximum throughput of 
5 units each quarter. From then on a company may add other 
production lines for $30,000 each. But each such $30,000 
Increment will Increase the maximum throughput by 5 . 

A company must pay for increased 
capacity as soon as It decides to start construction. 
Construction time Is nine months (three time periods), and 
only after completion may the first unit be put Into "work 
In progress" for the new production line. The companies 
are not allowed to sell or otherwise dispose of excess 
capacity. 

The total lead time In producing units 
In a company's plant Is six months. First, production Is 
scheduled, and this Involves no financial outlay. Then In 
the next quarter units are put Into "work In progress" and 
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must be paid for. In the subsequent quarter these units 
come off the production line , are added to Inventory, and 
may be sold. 

Total production cost contains a fixed 
cost and a variable element. The fixed cost Is Incurred 
each quarter, regardless of how many units are produced. At 
a maximum capacity of five units per quarter, the fixed cost 
Is $6000, and the variable cost per unit Is $3000. As 
capacity Is Increased by additional production lines, fixed 
costs rise and the variable cost per unit decreases. If a 
company, prior to adding a line, wants to know the exact 
costs It will Incur at the next level of capacity. It can 
get that Information from the umpires for $2000, but 
otherwise the umpires will Inform the company what 
production costs are when the new line goes Into production. 

Units are added to Inventory at actual 
cost. When a unit Is sold, however. It Is deducted from 
Inventory at the average cost ( total Inventory Investment 
divided by number of units In Inventory). 

The umpires should calculate the 
production costs at various capacity levels as follows: 

Max. capacity Total unit cost Fixed cost Variable cost 

per quarter per unit 
5 $i*,200 $6,000 $3,000 

10 3,600 n,<»00 2,200 

15 3,000 22,500 1,500 

20 2,i»00 28,800 1,000 

25 1,800 31,500 600 

Financial Management 

The management of a company's available 
capital Is of critical importance. Each company starts 
with $1*00,000 capital! and grow only through reinvested 
earnings. Profitability will be In direct relation to the 
skill with which the various parts of the business are kept 
In harmony with each other to achieve sound growth. 

The price per unit of product Is fixed 
at $10,000. When a sale Is made, accounts receivable are 
Increased by the total amount of the sale, and on the game 
board an accounts receivable symbol Is placed on the fifth 
space In the "accounts receivable" column. Every quarter 
this symbol Is moved up one space until after four quarters 
It reaches the top space and becomes cash. Competitive 
pressure In the Industry forces the extension of credit; 
hence the one year collection lag. 

If a company Is short of cash, accounts 
receivable may be factored to get cash Immediately. The 
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cost of doing this Is 20% of the amount factored. 
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Appendix B 

The following Is an MSL model of parts 
of the game (for SWS. "region") described In Appendix A— as 
seen from the point of view of a player wishing to 
Investigate the game and see the effects of various 
strategies. It Is presented here as an Illustration of the 
use of MSL. 



(♦ACTIVITY HIRING 

(♦PREREQUISITES (♦PRESENT (1000 CASH))) 

(♦SCHEDULE ON CALL) 

(♦PRIORITY 2) 

(♦OUTPUT (SOME TRAINEE)) 

(♦TAKES 0) 
) 

(♦ACTIVITY TRAINING 

(♦PREREQUISITES 

(AND (♦PRESENT (1000 CASH)) 

(♦PRESENT (SOME TRAINEE)))) 
(♦TAKES 3) 

(♦OUTPUT (SOME SALESMAN)) 
) 



(♦ACTIVITY URBAN-CALL 

(♦PREREQUISITES 

(AND (♦PRESENT (ASSIGNED 
(SOME SALESMAN) 
(SOME URBAN-CUSTOMER)) 
(♦PRESENT (5 00 CASH)))) 
(♦TAKES .5) 
) 

(♦ACTIVITY RURAL-CALL 

(♦PREREQUISITES 

(AND (♦PRESENT (ASSIGNED 
(SOME SALESMAN) 
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(SOME RURAL-CUSTOMER))) 
(♦PRESENT (1000 CASH)))) 
(♦TAKES 1) 

) 

(♦EVENT QUITTING 

(♦CONDITIONS QUITTING-PROBABILITY) 
(♦ACTIVITIES (SALES-CALL) 

(♦CANCEL) 

(♦REMOVE (THAT SALESMAN))) 
(♦ACTIVITIES (TRAINING) 

(♦CANCEL) 

(♦REMOVE (THAT TRAINEE))) 

(♦ACTIVITY ADVERTISING 

(♦PREREQUISITES (♦PRESENT (3000 CASH))) 

(♦SCHEDULE ON CALL) 

(♦OUTPUT (1 PAGE-OF-ADVERTISING)) 

(♦PRIORITY 3) 

(♦TAKES 1) 
) 

(♦ACTIVITY R&D 

(♦PREREQUISITES (♦PRESENT (10000 CASH))) 

(♦TAKES 0) 

(♦SCHEDULE ON CALL) 

(♦OUTPUT (10000 R&D)) 
) 



(♦EVENT PRODUCT- IMPROVEMENT 

(♦CONDITIONS P-I-PROBABILITY) 
(♦ACTIVITIES (R&D) 

(♦OUTPUT (1 PRODUCT- IMPROVEMENT))) 

(♦ACTIVITY PRODUCT-INITIATION 

(♦PREREQUISITES (♦PRESENT 

(1 PRODUCTION-LINE))) 
(♦TAKES 1) 
(♦OUTPUT (5 UNITS-IN-PROGRESS)) 

(♦ACTIVITY PRODUCTION-COMPLETION 

(♦PREREQUISITES (♦PRESENT 

(5 UNITS-IN-PROGRESS))) 
(♦TAKES 1) 
(♦OUTPUT (5 UNITS)) 
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) 



( * ACT I V I TY PRODUCT ION-LI NE-CONSTRUCT I ON 

(♦PREREQUISITES (♦PRESENT (30000 CASH))) 

(♦TAKES 3) 

(♦OUTPUT (1 PRODUCTION-LINE)) 

) 

(♦ACTIVITY FACTOR 

(♦PREREQUISITES (♦PRESENT (5000 A-R))) 

(♦TAKES 0) 

(♦OUTPUT (4900 CASH)) 

(♦SCHEDULE ON CALL) 
) 

(♦EVENT SALE 

(♦CONDITIONS SALES-PROBABILITY) 

(♦ACTIVITIES (SALES-CALL) 

(♦OUTPUT (10000 A-R))) 
) 

(♦FUNCTION SALES-PROBABILITY 

( ♦ARGUMENTS ( PAGE-OF-ADVERT I S I NG ) ) 
(PRODUCT- IMPROVEMENT)) 
(♦RETURN 

(♦SUM-UP 

.25 

(AD-FUNCTION 

PAGE-OF-ADVERTISING) 
(TIMES .10 

PRODUCT- IMPROVEMENT) 
)) 
) 

(♦FUNCTION AD-FUNCTION 

(♦ARGUMENTS (PAGE-OF-ADVERTISING)) 
(♦RETURN 

(♦TABLE (PAGE-OF-ADVERTISING 
♦RESULT) 
(0 0) (1 .04) (2 .10) (3 .17) 
(4 .23) (5 .27))) 
) 

(♦FUNCTION P-l PROBABILITY 

(♦ARGUMENTS (R&D)) 
(♦RETURN (♦TABLE (R&D ♦RESULT) 
((LESSP R&D 40000) 0) (40000 .02) 
(50000 .04) (60000 .07) (70000 .11) 
(80000 .15) (90000 .18) (100000 .20) 
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